We love using AWS Lambda. The simplicity, the scale, the speed - we find ourselves using it all the time (we are an AWS Lambda Service Delivery Partner, after all). We've also written about why you should use Infrastructure as Code (IaC) if you're working with the cloud. We find ourselves using Terraform more than any other IaC tool, and the intersection between Terraform and Lambdas can be confusing. This post will layout some thoughts on best practices for the lines of demarcation.
Like most things, this strategy will not always work. Imagine, if you will, needing to integrate a Lambda function with the Cognito Authentication flow. You need the Lambda ZIP in Terraform during execution, but you can't compile the Lambda without knowing details of the User Pool.... but we digress. This strategy works for the majority of cases!
To keep this brief- for everything except the API Gateway and the Lambda functions themselves. All other resources should be created via Terraform, and any required information can be passed to the Lambas as Parameters.
AWS provides the 'AWS Serverless Application Model' (AWS SAM) for just this case. AWS SAM is really just a wrapper for AWS Cloud Formation, but it provides additional functionality to aid in the development of Lambda functions. This blog won't dive into too much detail here, but a quick summary is that the SAM template can be used to run the functions locally (on Docker) for testing, as well as for deploying the lambda functions to the cloud. This template allows a simple configuration for the API Gateway and all Lambda functions in an understandable, repeatable manner.
Now that we know which technology to use for which task, how do we tie them together? I'll show a simplified example using AWS Code Pipeline.
We believe that this approach utilizes the strength of both technologies to provide the simplest solution. We could (and have) used Terraform or Cloud Formation independently to achieve the same final state, but have found that those solutions end up being inelegant and hard to maintain. Technology is supposed to make our jobs easier- and we believe that this approach is the easier solution.