Design S3 event consumers for duplicate and out-of-order delivery

Make object-event processing safe when notifications repeat or arrive late.

Source facts

S3 Event Notifications are designed for at-least-once delivery and do not guarantee event order. Retries can produce duplicate notifications for the same event. AWS also warns that writing processing output into a triggering bucket can create a loop unless the bucket or applicable prefix separates input from output.

Source: AWS Documentation.

Applicability

Use this when consuming S3 notifications with application workers, queues or Lambda functions.

DSE recommendation

DSE recommends defining an idempotent processing decision before connecting the event destination. Record enough object and event identity to recognize completed work, and decide how a later-arriving older event should be handled. Separate the output location from the trigger scope. Provide a controlled replay path that does not duplicate customer-visible effects. Keep processing failures observable without including confidential object contents in the diagnostic record, and assign an owner to review repeated or stalled events. Define duplicate handling at the business-effect boundary, such as the downstream record or customer notification. Ask the owner to decide how replay authorization works and which processing identifier support may disclose when investigating an apparently repeated event.

Verification

Proposed verification: replay the same disposable notification twice and then deliver an older event after a newer one. Confirm the intended final state and count externally visible effects. Verify that generated output does not match the input trigger and create another processing cycle.

Official reference

AWS Documentation. Reviewed October 20, 2026.