Redis SET NX EX to dedupe Sume webhooks by request_id

Short answer

Use SET with NX and an expiry on the request_id from the Sume webhook envelope. Per the Redis SET docs, NX sets the key only if it does not exist, and the reply is OK when it was set and nil when it was not. The first delivery gets OK and proceeds; a duplicate gets nil and is acknowledged without doing the work again.

Why duplicates happen

Sume retries a delivery that does not return a 2xx in time. Delivery makes up to 10 attempts with 10 seconds allowed for each, and a redirect counts as a failure. A slow handler that finishes after the attempt timeout produces exactly the duplicate you are guarding against: your code succeeded, Sume saw a timeout and sent the event again. The documented rule is to dedupe on the envelope request_id and order events by created_at.

Claim, then done, or release

Marking an event as seen before you process it has a trap: if the worker crashes after the claim, the retry sees nil and skips an event that was never handled. Use two states. Claim with a short expiry and the value processing. On success, overwrite with the value done and a long expiry; a plain SET discards the previous time to live, so the new expiry applies. On failure, delete the key so the next delivery can try again.

Choose the done expiry to outlast the retry horizon. Run webhook retries back off exponentially up to one hour between attempts, so a day is a comfortable minimum. Choose the claim expiry above your longest processing time, so a crashed worker's claim frees itself.

import os

import redis

r = redis.Redis.from_url(os.environ.get("REDIS_URL", "redis://localhost:6379"),
                         decode_responses=True)


def claim(request_id: str) -> bool:
    return bool(r.set(f"sume:wh:{request_id}", "processing", nx=True, ex=300))


def done(request_id: str) -> None:
    r.set(f"sume:wh:{request_id}", "done", ex=86400)


def release(request_id: str) -> None:
    r.delete(f"sume:wh:{request_id}")


if __name__ == "__main__":
    print(claim("req_demo"), claim("req_demo"))  # True False
    release("req_demo")

Where this sits in the handler

Do it after signature verification and after the fast acknowledgement path, usually in the worker that consumes your queue. Verify first, so unauthenticated callers cannot fill your cache with keys. Then claim, process and mark done. Return 2xx for a duplicate: a non-2xx would only make Sume send it again.

Order is a separate problem. A claim answers whether you have seen an event, not whether it is the newest. Compare created_at against the last applied event for the same run before overwriting state, because a retried older event can arrive after a newer one.

Limits of the pattern

The Redis docs describe the SET NX EX recipe as a simple lock and note that it is discouraged for strong locking guarantees in favor of Redlock. For dedupe that matters less, since the cost of a rare double process is low if your handler is idempotent anyway. Treat the cache as an optimization over an idempotent write, not as the only line of defense: key your database write on the same request_id with a unique constraint.