Webhook events arrive out of order: Sume sends one event per job

Do not write a webhook handler that depends on event order. Stripe's webhook guide says it does not guarantee that events arrive in the order they were generated, and it tells you not to use the created timestamp to decide order or whether you already processed an event. Sume avoids most of the problem by design: a generation job webhook is a terminal event only (job.completed, job.failed, job.canceled), with no progress or partial deliveries. One job produces one terminal state, so the only ordering question left is duplicates and late arrivals.

What can still go wrong with a terminal-only event

Even with one event per job, three things happen in practice. A retry can deliver the same event twice. A poll can see completed before the callback lands. And after ten refused attempts the job is still terminal, but your system never got told. Sume's docs say to use job_id as the idempotency key on your side, and to keep status_url polling as a backup because delivery is an optimization, never the only recovery path.

Stripe gives the same advice from the other direction: log the event IDs you have processed, and retrieve the object from the API when an event shows up before the thing it depends on.

• Key every write on job_id (job webhooks) or request_id (run webhooks), never on arrival time.
• Treat the webhook as a hint to read the job, not as the source of truth: GET /v1/jobs/{job_id}/status is authoritative.
• Make the handler safe to run twice. The second run should change nothing.
• Return a 2xx only after the event is stored durably.

A handler that ignores order

The sketch below records the first terminal event for a job and ignores every later copy. Swap SQLite for your own store. The point is the primary key on job_id.

import sqlite3

db = sqlite3.connect(":memory:")
db.execute("create table seen(job_id text primary key, status text)")

def first_delivery(event: dict) -> bool:
    cur = db.execute(
        "insert or ignore into seen values (?, ?)",
        (event["job_id"], event["status"]),
    )
    db.commit()
    return cur.rowcount == 1

event = {"event": "job.completed", "job_id": "job_123", "status": "OK"}
print(first_delivery(event))  # True: process it
print(first_delivery(event))  # False: duplicate, skip

Stripe and Sume side by side

Build the poller before you need it

A cron that lists your open jobs and reads their status costs little and closes the gap that no retry policy closes. Do not resubmit a paid job because a callback was late: the job is already running and billing, and resubmitting only creates a second charge.

Why the status field helps you too

Sume's job webhook body carries status: "OK" for a completed job and status: "ERROR" plus an error object for failed and canceled jobs, alongside event, request_id and job_id. That makes a first-pass handler simple: branch on event, write the row keyed by job_id, and enqueue the real work for later. Do not fetch media, call a model or update an invoice inside the request, because the delivery attempt has a 10-second timeout and a slow endpoint spends the retry budget.

Run webhooks for Formats, Actions and Agent Completions have a different payload (a full run receipt) and a different event name (format.run.terminal and its siblings). The signature scheme is identical, so the verifier you write once covers both, but the dedupe key there is request_id. Mixing the two keys in one table is a common source of silent duplicates.