Using SQS with Lambda and Dynamodb

In this post, you learn how to use AWS CDK to create a backend system which persists data to a database using SQS and Dynamodb and Lambda. Although this tutorial uses serverless components, it might incure costs in your account, so don’t forget to run “CDK destroy” at the end.

First thing’s first, let’s initialize an empty CDK typescript project: cdk init –language=typescript.

Then we go to package.json to declare the dependencies for our project. Add the following modulesto the dependencies:

"@aws-cdk/core": "*",
"@aws-cdk/aws-dynamodb": "*",
"@aws-cdk/aws-lambda": "*",
"@aws-cdk/aws-lambda-event-sources": "*",
"@aws-cdk/aws-sqs": "*",

This is an optional step, but go ahead and run “npm install” in the root directory of your project. This will download the dependencies and give you intellisense which comes very handy. Then go to the lib folder and the typescript file for your application stack.

Add the following import statements to before class declaration:

import * as cdk from '@aws-cdk/core';
import * as sqs from '@aws-cdk/aws-sqs';
import * as lambda from "@aws-cdk/aws-lambda";
import * as dynamodb from "@aws-cdk/aws-dynamodb";
import { SqsEventSource } from '@aws-cdk/aws-lambda-event-sources';
import { Duration } from '@aws-cdk/core';

We begin by creating our Queue:

const queue = new sqs.Queue(this, "Orders", {
  queueName: "Orders",
  retentionPeriod: Duration.days(4)
});

using the retentionPeriod parameter you could specify how long your message stays in the queue, for up to 14 days. You could also add deduplication, Fifo…

Then we instantiate our DynamoDB Tables.

const dynamoTable = new dynamodb.Table(this, 'OrdersDB', {
  partitionKey: {
    name: 'itemId',
    type: dynamodb.AttributeType.STRING
  }
});

Here also you could specify encryption, removalpolicy and many other options.

Then by creating our lambda function we bring everything togher:

const createLambda = new lambda.Function(this, 'createItem', {
  code: new lambda.AssetCode('./src'),
  handler: 'create.handler',
  runtime: lambda.Runtime.NODEJS_10_X,
  environment: {
    TABLE_NAME: dynamoTable.tableName,
    PRIMARY_KEY: 'itemId'
  },
  events: [
    new SqsEventSource(queue)
  ]
});

Notice the events portion, where we tell our lambda to listen for SQS events.

Almost there! At this point our lambda cannot talk to DynamoDB, because it doesn’t have the required IAM roles. We go ahead and generate that:

dynamoTable.grantReadWriteData(createLambda);

Now we are ready to verify our cdk stack. First run ‘npm run build’ which compiles typescript to javascript. Then run cdk synth. This should give a cloud formation template. If it returns an error, go to previous steps and verify your work. Before you can deploy this applicaion, you need to bootsrap your environment. To do that, run the following.

cdk bootstrap aws://youraccountID/yourregion

set CDK_NEW_BOOTSTRAP=1
cdk bootstrap aws://youraccountID/yourregion

Now you are ready to deploy your cdk app.

cdk deploy

Some more useful cdk commands:

cdk ls //lists the stacks in your application
cdk diff //shows the difference of your current deployed stack and your local stack
cdk doctor //*kind of* verifies your cdk stack for warnings

You could use such cron jobs for sending newsletters, verifying state of your application…

The source code on github: https://github.com/pedramha/cdk-sqs-lambda