AWS Lambda announces new capabilities to optimize costs up to 90% for Provisioned mode for Kafka ESM

AWS Lambda Provisioned Mode for Kafka ESM Updates

AWS Lambda introduces new capabilities for Provisioned mode for Kafka event source mappings (ESMs), enabling cost optimizations of up to 90% for Kafka ESMs. These enhancements allow grouping of Kafka ESMs and support for higher density of event pollers, providing benefits such as throughput controls, schema validation, event filtering, low-latency invocations, and enhanced error handling.

With the new Event Poller Unit (EPU) configuration, each EPU now supports up to 10 event pollers by default for low-throughput use cases, improving EPU capacity utilization. Multiple Kafka ESMs can be grouped within the same Amazon VPC to share EPU capacity using the PollerGroupName parameter. These optimizations are available in all AWS Commercial Regions where Provisioned mode for Kafka ESM is offered.

What to do

• Configure the PollerGroupName parameter to group Kafka ESMs and share EPU capacity.
• Adjust minimum and maximum event poller settings to optimize costs.
• Implement ESM grouping through the Lambda ESM API, AWS Console, CLI, SDK, CloudFormation, and SAM.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.