Amazon DynamoDB Streams now supports timestamp-based stream processing

Amazon DynamoDB Streams Timestamp Processing

Amazon DynamoDB Streams now supports timestamp-based stream processing, allowing you to start reading stream records from a specific point in time. Previously, you could only start from the oldest retained record (up to 24 hours old) or from new records written after you start consuming the stream. Now, you can supply a timestamp and begin processing at the first record written at or after that time using the AT_TIMESTAMP shard iterator type.

This feature can be used with the Amazon Kinesis Client Library (KCL) through the DynamoDB Streams Kinesis adapter, helping you avoid reprocessing entire retained streams or missing records when resuming stream consumption. Common use cases include recovery after an interruption, data replay for debugging and auditing, skipping a backlog by resuming from a recent timestamp, and synchronizing after a restore by beginning at the export timestamp following a point-in-time recovery (PITR) export.

What to do

• Review the Amazon DynamoDB Streams timestamp documentation in the DynamoDB developer guide.

Source: AWS release notes

If you need further guidance on AWS, our experts are available at AWS@westloop.io. You may also reach us by submitting the Contact Us form.