We love tools

Doesn’t matter if that’s IntelliJ or Eclipse. Many of us cannot imagine programming without the use of that the-only-one-in-the-world-having-everything-built-in IDE. You probably already know the plugins that help you integrate the IDE with an application server like JBoss or Tomcat. But we are looking higher! We are looking into clouds! And that’s why in this post I want to introduce Sigma IDE - a cloud IDE, which helps you build, test and deploy serverless applications.

Do you remember our recent post about the Serverless Cat Detector Workshop? I am happy to announce that the source code of the project is now publicly available. You can find it in our official Github repo.

Like it, hate it, share it! Experiment as much as you want! You will find a detailed instructions on how to run it using Serverless Framework.

In this article, I will give Sigma IDE a chance to show its awesomeness in running our Cat Detector application! Stay tuned!

It’s a cloud-based Hybrid IDE, they say

Take a look at the page. Sigma is an IDE that supports developing serverless applications thanks to the following features:

  • drag-n-drop composition of the application
  • code completion
  • one click deployment
  • VCS integration
  • live debug

Sounds great, doesn’t it? But I am not a salesman, I am a programmer. And this is their promise I want to test:

Drag-n-drop service entities and function triggers right into your code, and let Sigma take care of the configuration and integration burden for you. A fairly simple, intuitive environment, with easy, drag-and-drop composition combined with the full power of written code.

Let’s do it!

Signing up

Just sign up for a free account. It’s enough for testing. After first log-in it will ask for your AWS credentials. You can also store it in the cloud and change anytime you want.

You can ask what role is required for your AWS user. Well, as I got informed, for now it’s recommended to give it an admin. On the one hand it seems to be risky. On the other hand you will quickly find a reason. Once the IDE is opened, it will create the whole infrastructure in your AWS account that is needed for the IDE to work properly. This is to simplify creating and testing the application without the real deployment.

Taking into account that after this initial step the whole IDE worked smoothly, I almost forgot about that little detail of granting admin privileges to it.

Nothing is free.

Before going further, I want to tell you about the great support that Sigma’s crew provides! The IDE has live chat built-in and the guys answer nearly real-time. Moreover, they are true developers familiar with the product they sell and able to help you with the issues. Guys, chapeau bas!

Naive way - importing the project

The Sigma IDE has an option to import the project from the Serverless Framework model. I tried it but it failed. Basically for two reasons:

  • the project uses custom configuration file ${file(src/config.js)}, which is not yet supported
  • there are dependencies between the app components that the IDE is not able to read from Serverless Framework model

Ok then. Let’s give it another try.

What we gonna build

As currently we are not able to just import the project, let’s implement its core part - image recognition. The application will consist of four components, as shown below:

  • S3 storage - in order to perform cat recognition, user sends an image there
  • Lambda - triggered by the new object in S3 storage will send the image to the Rekognition service in order to label the picture
  • Rekognition - labels the image in order to classify it - if there is a cat in it or not
  • DynamoDB - stores the results of image classification - if the picture contains a cat or not
Rekognition flow - the part that will be implemented in the Sigma IDE

Notice that we already have the whole implementation. Hopefully we will be able to copy it to the Sigma IDE and run the application.

Start with copying required files

Start with copying the following files from the repo:

  • classification.js - the code of the Lambda Function
  • recognition.js - the module responsible for image labeling
  • persistence.js - the module responsible for database operations
  • config.js - the module containing configuration properties

At this point you should be able to build and deploy the project. VCS integration is required to store the files.

Actually the project does nothing because of the red point you can see in the editor.

Sigma IDE detects that no trigger is set for the Lambda Function

Sigma IDE detects that no trigger is defined for the Lambda Function. You can drag-n-drop one of the services displayed on the left pane.

So far so good.

Define S3 trigger

Now the tricky part. Drag the S3 icon from the left pane and move it just below the handler definition. You should see that the event param is highlighted and S3 event configuration form is shown.

Sigma IDE simplifies trigger configuration

Choose New Bucket, name it and subscribe for Object created events.

This is quite a nice feature as two things happen under the hood:

  • S3 bucket will be created with the name that you specified
  • Lambda function will be configured to subscribe for new object creation event

Build and deploy the application.

Sigma IDE shows the CloudFormation change set and then monitors update progress

You should see the changes that the IDE is going to introduce. Like a Change Set that you would normally do in the CloudFormation.

If everything went well, you can verify the service in the AWS console.

AWS console confirms that S3 bucket is crated and Lambda trigger is defined as expected

That was easy! Basically we have everything to try it end-to-end.

First end-to-end test

Open AWS console, navigate to S3 service and then open the bucket that you defined in the project. Upload a picture. Then navigate to the Lambda Function and check the logs.

Digging into that, you can see the app is not able to load aws-xray-sdk module. Yeah, I didn’t tell you anything about managing dependencies in the project, did I?

You just need to search for npm dependency that you want and Sigma IDE will automatically package it into deployment archive while deploying the project.

Sigma IDE manages npm dependencies for you

Easy-peasy! Build it, deploy and give it another try!

Dependencies are important but spelling is importanter

After running another test you probably faced the following.

Sigma IDE requires a specific handler name

As I get informed, for now the IDE requires the lambda function to have handler name. Ok, let’s change module.exports.imgClassification to module.exports.handler in the classification.js file.

And… give it another try! We are getting closer!

Who am I

After fixing the name, let’s do another thing. Instead of running the application again, we will use Sigma IDE built-in testing feature.

This is almost the same as the one available in the AWS console. Please copy the example request to the Lambda Function from CloudWatch logs or modify the one below.

Run it and see the output logs at the bottom. Sigma IDE has really a nice feature - that you can use it to check the logs of lambda you are working on. Additionally:

  • logs that are a result of execution from IDE are marked with [TEST] tag
  • logs that are a result of execution in AWS are marked with [PROD] tag
  • you can enable real-time log data fetching from the AWS

So, what do we have?

This is the output from the IDE.

Sigma IDE provides a nice feature for watching the logs

And this is the output from the AWS.

Sigma IDE provides a nice feature for watching the logs

Can you spot a difference?

Digging into lambda code, you can notice that the classification result is put into database twice:

  • first when processing starts - new file registered
  • second time at the end of the execution - recognition result updated

Comparing the logs we can see that the test execution tried to register new file in the database (but failed because the table doesn’t exist yet), while the prod execution failed because of the authorization.

The idea that came to my mind is that for testing purposes, Sigma IDE uses execution role that is associated with the AWS credentials that we provided. On the other hand, prod execution in the AWS itself, uses the role that is specified for the Lambda Function. I confirmed it with the guys on the support.

So we have two issues now:

  • DynamoDB table is missing
  • Lambada execution role requires more privileges

One more step.

CloudFormation for rescue

At this point the Sigma IDE won’t help you much because:

  • you cannot define resources that are not supposed to trigger your lambda function
  • as a consequence, lambda execution role will be missing the policies that are required to use such a resource if you add it another way

The only thing we can do, is to modify CloudFormation template, which is possible in the Sigma IDE.

Let’s fix the two issues at once using the CloudFormation snippet.

Copy and paste it into Deployment Template Editor.

Sigma IDE allows to modify CloudFormation deployment template

With this snippet we are adding DynamoDB table and modifying the lambda execution role to allow Dynamodb, Rekognition and S3 operations.

Build, deploy and test. Should work now! To double check it, open AWS console, go to DynamoDB, then open your table and see the items.

Debugging, debugging, tracing

At this point we have running application. But let’s check another feature in the Sigma IDE which is live debugger.

That is as awesome as easy to use! Just enable debugger in the IDE and copy the URL.

Sigma IDE has built-in live debugger

And open it in another tab in your browser.

Sigma IDE debugger in action

Then get back to the IDE and run a test to start debugging.

You may also find an X-ray instrumentation in the source code. This is something that also needs to be enabled in the Lambda Function, but Sigma IDE doesn’t support it so far. The option here is to modify the CloudFormation template again.

Takeaways

Thank you for that journey! Hope that this practical example will help you to build your serverless applications and made you more aware of the Sigma IDE features that may help you.

My findings regarding the IDE in a nutshell:

  • It still needs some improvements to support Serverless Framework applications. If they decided to offer the import function, they will probably improve it.
  • Creating a new Lambda Function and defining its trigger is really nice and easy. It’s a huge help for the people that are not really familiar with AWS.
  • Also, building and deploying the project is painless.
  • Testing framework looks good, but for sure it should use the same execution role as a target application.
  • Be careful about the naming of the handler.
  • But hopefully you can see any error in the built-in logs pane. Adding real-time log fetching from the AWS makes it really nice feature.
  • For now, it is not possible to freely define any AWS resource that you want. But you can always modify CloudFormation deployment template.
  • Still not all the AWS Lambda properties are available in the IDE. For example you cannot enable X-Ray tracing.
  • Speaking about tracing, the IDE has a lovely debugger built-in.
  • Last but not least, Sigma IDE crew provides awesome live support! They answer nearly real-time and know the product they sell!

I really enjoyed working with Sigma IDE. Although there are still many areas to improve I can recommend it to those who are just starting their serverless development. Also, more advanced users will find some of the features really helpfull, even that sometimes they may need to modify CloudFormation template manually.

Think about the business and leave technology for us

We are always up-to-date with the technology that we use. Take advantage of our knowledge. Compete with others on the idea level. We win technology for you.

Estimate my project

Nitpicking

It might happen that after getting back to the IDE with the opened session you will face the issue that the deployment package doesn’t include all the project files. Just refresh the window and it should work. Of course I asked! This is a known issue and the guys are already working on it.