AWS Lambda supports OAuth authentication for self-managed Apache Kafka event sources

AWS Lambda OAuth Support for Kafka

AWS Lambda now supports OAuth authentication for self-managed Apache Kafka event source mappings (ESM), including Kafka clusters that customers run themselves and managed offerings such as Confluent Cloud, Aiven, and Redpanda. This allows customers to authenticate their Lambda Kafka consumers through their enterprise identity provider, meeting security and compliance requirements for Kafka applications.

Customers can now build event-driven Kafka workloads for use cases such as payment processing, fraud detection, and real-time data pipelines using Kafka ESM. Kafka ESM provides automatic scaling, error handling, batching, and event filtering. Previously, Kafka ESM supported only SASL/PLAIN, SASL/SCRAM, and mutual TLS (mTLS) as authentication methods, but now OAuth support is available.

What to do

• Create a new Kafka ESM with your authentication configuration through the AWS Management Console, Lambda API, AWS CLI, AWS CloudFormation, or AWS SAM.
• Use your enterprise identity provider, such as Amazon Cognito or Okta, to authenticate your Kafka ESM.
• Apply the same identity governance and access policies across your Kafka clusters and Lambda consumers.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.