Introduction

This blog post is aimed for people who are starting to use AWS Lambda with Java and who would like to save some time by getting knowledge about a few interesting things related to Lambdas, asynchronous invocations and error handling.

We will be using AWS Lambda service with Java services using AWS Java SDK 2.0. We will create 2 Lambda functions. The first one will asynchronously invoke the second one. Along the way we will create proper and minimal permissions for these operations. Finally we will see what happens when errors strike and how to deal with them. We hope it will be really helpful and time-saving for anyone that has just started and is having a hard time understanding all the nuts and bolts of using Lambdas from Java.

You can grab a small sample project that will be used in this blog post on our GitHub Repo.

Prerequisities

We are assuming that you already have an AWS Account with programmatic and console access.

Make sure you have aws-cli (command line interface) installed. You can see details here.

For deployments we will use AWS SAM (Serverless Application Model).

Finally, for Java development and some testing it will be good to have Java environment variables set up and have Maven installed.

Setting up the project

You can grab complete project from our GitHub Repo or if you want to start quickly you can bootstrap Java Lambda project using official Amazon Maven archetype

In the projects directory we can generate new project:

It will ask you for a bunch of properties. Provide group (package) and artifact (project name) Ids as you wish. As a region you can put any region that is close to you. We have used eu-west-2 for London. For “service” property write “lambda”. And that’s it. You should have a sample project with just one Lambda function and SAM template. Traverse into project directory and run mvn clean install to build and create target jar file. If output of this command shows that 0 tests were run, make sure you have Maven in at least 3.6+ version and in your pom.xml in <build> section there is a configuration for surefire plugin that should work nicely with jUnit 5:

After that on mvn clean install or mvn test you should see 1 test pass and we are ready to go. We have commented this single test because this blog post is not about testing and we don’t want to focus on this particular aspect.

Creating two Lambda functions

We need 2 functions for demonstration purposes. Either check our example project on GitHub or create 2 classes:

public class FirstFunction implements RequestHandler<Object, Object>

and

public class SecondFunction implements RequestHandler<Object, Object>

with implementation of AWS Lambda required method:

  @Override
  public Object handleRequest(final Object input, final Context context) {

The FirstFunction will need to asynchronously invoke SecondFunction, so it will need to have some AWS Lambda SDK classes. Fortunately required dependencies are already present in Maven pom generated from AWS archetype.

Adding asychronous invoke code

You can easily fall into some trap here. When checking out AWS Java SDK 2.0 documentation you will come upon interface called LambdaAsyncClient. This looks like what we need here, right? That’s not the correct interface however. LambdaAsyncClient provides us a way to run our Java call on SDK method in asynchronous way. In other words it immediately returns CompletableFuture turning our code to asynchronous. This has its use cases, for example when we really want to fan-out our operations to do as much simultaneous work as possible in the shortest time, but this is not what we want to achieve here. What we really want is to invoke the SecondFunction without waiting for AWS to return us full status and response, so waiting for the SecondFunction Lambda to finish completely. In this way our FirstFunction can finish while SecondFunction still runs, possibly for much more longer time, doing some background work. To get there, we don’t need LambdaAsyncClient but only LambdaClient. Sure, the call to invoke is synchronous, so our Java thread will wait for method return, but we must tell AWS that we want to invoke Lambda asynchronously. How? By passing parameters:

We are preparing a very simple json input object as payload and the name of SecondFunction. The most important part is invocationType(InvocationType.EVENT). This enum tells AWS that we (the caller) want to invoke SecondFunction in asynchronous way. This call will return quickly with 202 (Accepted) and without any response (payload). AWS will take care of calling SecondFunction and our FirstFunction can continue or exit.

SecondFunction is very easy for our purposes:

It just logs that is has been called and dumps input to the console. These logs will be passed to CloudWatch so we could see them.

Preparing for deployment and adding proper permissions

Ok, let’s test this. We will use SAM to deploy everything. Check template.yaml file here. It is pretty straightforward, but for now let’s focus on definitions of 2 Lambdas: FirstFunction and SecondFunction. We need to have correct entries in “Handler” properties pointing to our classes and handleRequest methods. FirstFunction must have permissions to call SecondFunction:

  Policies:
    - LambdaInvokePolicy:
        FunctionName:
          !Ref SecondFunction

By default SAM will also add permission to write Lambda logs to CloudWatch for both functions. This is very handy and crucial for us to see what is going on.

So this SAM template will be automatically transformed into CloudFormation template and applied to our AWS resources, configured by aws-cli and environment variables giving programmatic access.

Let’s build by mvn clean install and deploy by sam deploy --guided. SAM will ask you for a couple of properties and later on you will only need to run sam deploy to deploy changes without any hassle. Deployment may take a couple of seconds and when it is done you can visit your AWS Lambda service. You should see 2 functions:

Two Functions

You can click on FirstFunction and check if it indeed has correct permissions:

Permissions

Testing

While still on AWS Lambda page, go to FirstFunction and in the upper right corner add new test event:

Test Event

You can leave the event as the default one:

Test Event

And save it. Then hit the “Test” button. Initially your Lambdas are COLD so first call may take up to 3 seconds. But it should be successful:

Successful Call

But what is more important is to check if SecondLambda has been invoked. Go to the SecondFunction -> Monitoring -> “View logs in CloudWatch”. You should see our simple log:

Successful Call

For additional testing you may want to add some long running operation (like Thread.sleep(45000);) in SecondFunction to be really sure that FirstFunction was not waiting for completion of SecondFunction.

When something goes wrong

Ok. So now everything is working fine. But what happens if SecondFunction ends with an error? We know that the caller (FirstFunction) gets 202 when AWS acknowledged async invoke request and that’s all. FirstFunction will not know whether Second did its job or failed. If we leave everything as it is now, AWS will retry SecondFunction invocation 2 times, after 1 minute and then after additional 2 minutes. It will use the same input object as was specified in the call from FirstFunction. If these 2 retries fail, there will be no more retries. The only indication of the error will be logs in the CloudWatch and Lambda metrics for error invocation going up. To simulate this you can add artificial Exception throw in SecondFunction:handleRequest:

Then do the test call from FirstFunction and observe CloudWatch logs of SecondFunction. Give it ~4 minutes to see 3 calls with the same input object each of them failing with your Exception.

We can do better. Each asynchronously invoked Lambda can use Dead Letter Queue service to put failed invocations into. It can be SNS or SQS. Let’s configure SNS with email to let us know when asynchronous invocation really failed and will not be retried anymore.

SNS and email notifications for failed invocations

First, let’s create SNS Topic and attach it to our SecondFunction. We can use SAM template to achieve this:

  SecondFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: SecondFunction
      Runtime: java8
      Handler: com.pattermatch.aws.lambda.java.SecondFunction::handleRequest
      Timeout: 60
      MemorySize: 512
      CodeUri: ./target/asyncinvoke.jar
      DeadLetterQueue:
        Type: SNS
        TargetArn: !Ref AsyncInvokeErrorsTopic
  AsyncInvokeErrorsTopic:
    Type: AWS::SNS::Topic
    Properties:
      TopicName: "AsyncInvokeErrorsTopic"
      Subscription:
        - Endpoint: your.email@email.com
          Protocol: "email"

AsyncInvokeErrorsTopic is the SNS topic. Please put your valid email in “Endpoint” property. Then in SecondFunction DeadLetterQueue is configured to point to this topic. SAM will take care of updating permissions so we don’t have to worry about that. Run sam deploy to deploy changes and check your email afterwards. You should receive email with link on which you need to explicitly click to subscribe to this SNS topic. Then rerun test call from FirstFunction and check CloudWatch logs for SecondFunction. It will retry 2 times but when 3rd call fails you will receive email with input payload used from FirstFunction call. This is very important and very powerful mechanism because you can easily extend it to automatically invoke another Lambda to run cleanup or retry when you had a chance to fix the error.

Wrapping up

Please see our Github Repo project for a complete working example. We have shown how to quickly bootstrap Lambda Maven project, deploy it using SAM then asynchronously invoke one Lambda from another and finally how to capture errors. If you would like us to extend this example or provide another one related to this topic please let us know in the comments. Cheers!

Efficient Cloud That Suits Your Pocket

We have many years of experience with migrating, designing and optimizing cloud systems. Let us prepare a solution that suits your needs.

Schedule a call with our expert