AWS Lambda announces enhanced error handling capabilities for Kafka event processing

AWS Lambda Enhanced Error Handling for Kafka Event Sources

AWS Lambda has introduced enhanced error handling capabilities for Amazon Managed Streaming for Apache Kafka (MSK) and self-managed Apache Kafka (SMK) event sources. This update allows customers to build custom retry configurations, optimize retries of failed messages, and send failed events to a Kafka topic as an on-failure destination, enabling resilient Kafka workloads with robust error handling strategies.

What to do

• Configure Kafka ESM in the ESM API, AWS Console, or AWS CLI.
• Define specific retry limits and time boundaries for retries.
• Set automatic retries of failed records in the batch.
• Enhance function code to report individual failed messages.

This feature is available in all AWS Commercial Regions where AWS Lambda’s Provisioned mode for Kafka ESM is available.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.