A Pragmatic Way of Developing Workflows with Step Function and AWS CDK

In this post we are going to take a pragmatic approach on how to develop workflows with step function and AWS CDK. In previous posts (and also many examples on github) you can see how cdks StateMachine and Chain constructs can be used to implement workflows. However, here, we take a different approach. As you might already know, AWS has recently added a new capability to step functions, namely the possibility to define workflows visually. Something Azure Logic does as well. Firstly, let us see why we use such tools like step functions, and why I think that the approach that we are going to take is a pragmatic one. Generally speaking, one of the reasons that you might be interested in step functions as a developer is defining workflows in this fashion facilitates communication between you and your non-technical colleagues, e.g. business analysts. This way you can make sure that you are on the same page regarding the business logic. CDK offers two types of construct, level 1 and level 2. Level 1 constructs mostly begin with a cfn and are basically a wrapper for a cloud formation template. Level 2 constructs are more high level and tend to abstract a lot of the boilerplate code. In most cases, the level 2 constructs are more convenient to use, however, with step functions, IT DEPENDS! Depending on how you develop your workflows and work with your team, you might be better of using level 1 constructs. As mentioned earlier, AWS Step Function now has the capability of defining state machines visually. Moreover, In using aws toolkit VS Code Extension, you can define state machines in ASL (amazon state language) and visualize them in the editor. This way you have a common medium of communication with your business analyst counter parts. After defining the workflow in ASL, you can use cdk level 1 constructs to deploy the state machine with the least amount of overhead. Now, let us implement an example.

So as always, first thing’s first, let’s initialize an empty CDK typescript project:

cdk init –language=typescript.

Then we go to package.json to declare the dependencies for our project. You need the following dependencies:

"@aws-cdk/core"
"@aws-cdk/aws-apigateway"
"@aws-cdk/aws-lambda"
"@aws-cdk/aws-iam"
"@aws-cdk/aws-dynamodb"
"@aws-cdk/aws-stepfunctions"
"@aws-cdk/aws-stepfunctions-tasks"

So please either use npm to install them or use package.json to declare them.

Then import:

import cdk = require("@aws-cdk/core");
import apigateway = require("@aws-cdk/aws-apigateway");
import dynamodb = require("@aws-cdk/aws-dynamodb");
import lambda = require("@aws-cdk/aws-lambda");
import { RemovalPolicy } from "@aws-cdk/core";
import { BillingMode } from "@aws-cdk/aws-dynamodb";
import * as stepfunctions from "@aws-cdk/aws-stepfunctions";
import * as iam from "@aws-cdk/aws-iam";
import * as fs from "fs";

The “fs” import is used to import the asl file declared separately. As mentioned, using level 1 construct means also writing a bit of boilerplate code. Also, you can go ahead and create an asl file (something like workflow.json.asl) in your folder, and reference it when we create our workflow. So we begin with creating IAM Role:

const roleARN = new iam.Role(this, 'StepFunctionRole', {
  assumedBy: new iam.ServicePrincipal('states.amazonaws.com'),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole'),
    iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaVPCAccessExecutionRole')
  ]
});

Then we read the asl file (I created a folder called ‘logic’ in the root of my project folder):

const file = fs.readFileSync('./logic/stepfunction.json.asl', 'utf8');

Then we declare our state machine with the asl file and the role we declared.

const cdfnStepFunction=new stepfunctions.CfnStateMachine(this, 'cdfnStepFunction',
{
  roleArn: roleARN.roleArn,
  definitionString: file.toString(),
});

At this stage, we will need an api gateway and a lambda function to invoke our state machine. So let’s go ahead with the lambda.

const stepFuncStarter = new lambda.Function(this, "StepFuncHandler", {
  runtime: lambda.Runtime.NODEJS_14_X,
  code: lambda.Code.fromAsset("./src"),
  handler: "index.handler",
  timeout: cdk.Duration.seconds(10),
  environment: {
    STEPFUNCTION_ARN: cdfnStepFunction.attrArn
  },
});

Then, I create a folder called ‘src’ in my project root directory, and add a file called ‘index.js’. I add the following code to the js file:

    const AWS = require('aws-sdk');
    const stepfunctions= new AWS.StepFunctions();
    const STEPFUNCTION_ARN = process.env.STEPFUNCTION_ARN;
    exports.handler = async (event = {}) => {
    const ID = String.fromCharCode(65 + Math.floor(Math.random() * 26));
        var params = {
            stateMachineArn: STEPFUNCTION_ARN,
            input: "{\"your input\" : true}",
            name: ID,
            traceHeader: 'test'
          };
          console.log(params);
        try {
            //invoke stepfunction
            const data = await stepfunctions.startExecution(params).promise();
            //return 400
            return { statusCode: 200};
        }
        catch (stepfunctionsError) {
            return { statusCode: 500, body: JSON.stringify(stepfunctionsError) };
        }
    };

Don’t forget to add the policy:

stepFuncStarter.addToRolePolicy(new iam.PolicyStatement({
  actions: ["states:StartExecution"],
  resources: [cdfnStepFunction.attrArn]
}));

Finally let’s create our API gateway:

//create a api gateway
const api = new apigateway.RestApi(this, "StepFuncApi", {
  restApiName: "StepFuncApi",
  description: "StepFuncApi",
  endpointTypes: [apigateway.EndpointType.REGIONAL]
});

//add api gateway resource
const resource = api.root.addResource("orders");
const stepFuncIntegration = new apigateway.LambdaIntegration(stepFuncStarter);
resource.addMethod("POST", stepFuncIntegration);

Now the moment of truth:

CDK synth

Now we are ready to verify our cdk stack. First run ‘npm run build’ which compiles typescript to javascript. Then run cdk synth. This should give a cloud formation template. If it returns an error, go to previous steps and verify your work. Before you can deploy this applicaion, you need to bootsrap your environment. To do that, run the following.

cdk bootstrap aws://youraccountID/yourregion

set CDK_NEW_BOOTSTRAP=1
cdk bootstrap aws://youraccountID/yourregion

Now you are ready to deploy your cdk app.

cdk deploy

Some more useful cdk commands:

cdk ls //lists the stacks in your application
cdk diff //shows the difference of your current deployed stack and your local stack
cdk doctor //*kind of* verifies your cdk stack for warnings

That is it. Thank you very much for reading this post.

Github repo: https://github.com/pedramha/cdk-stepfunction