Cloudflare K2 as a buffer in front of Sume bulk runs

Yes, a Cloudflare K2 stream works as the queue that sits between whatever creates video requests and Sume's bulk runs. Producers append one event per request, a single consumer pulls a batch, posts it to POST /v1/formats/{handle}/{slug}/bulk-runs, and acknowledges the batch only after Sume answers 202.

The part that needs care is redelivery. K2 can hand the same batch to a consumer twice, so the Sume call must be safe to repeat. This post covers the ack, nack and park decision, and the one case where Sume's idempotency does not save you.

What does K2 promise, and what does Sume promise back?

Cloudflare's announcement describes K2 as serverless event streaming built on R2, in public beta for Workers Paid. Consumers pull batches through subscriptions, a failed batch can be nacked for redelivery, and a batch is leased for five minutes. Sume's bulk endpoint is built to take the same batch twice if you give it a stable key.

How should the consumer decide between ack, nack and park?

Decide from the HTTP status, not from guesswork. A 202 means the queue exists, so ack. A 429 or 503 means the request never became a queue and a later attempt can succeed, so nack. Everything else, such as 400 invalid_request, 403 or 409 idempotency_conflict, will fail the same way every time, so a nack would only loop the batch until you notice.

• Ack on 202. Store the returned queue id (frq_...) next to the event ids so you can poll it later.
• Nack on 429, 503 and network errors. For 429, wait the retry-after seconds; bulk create spends the write budget.
• Park on 400, 403 and 409. For a 400, details.index names the bad item: move that event aside, then resubmit the rest. Because K2's announcement does not describe a dead-letter queue, keep your own parked-events store.
• Never ack before the 202. If the consumer dies after posting but before acking, the redelivery replays the same key and gets the old queue back.

What is the submit function?

The key is derived from the sorted event ids, so a redelivery of the same batch produces the same key and the same payload.

import hashlib, os, requests

API = "https://api.sume.com/v1/formats/myteam/product-promo/bulk-runs"

def submit(events):
    # events: [{"id": "evt_1", "item": {"instruction": "...", "input": {...}}}]
    events = sorted(events, key=lambda e: e["id"])
    key = hashlib.sha256("|".join(e["id"] for e in events).encode()).hexdigest()
    headers = {
        "Authorization": "Bearer " + os.environ["SUME_API_KEY"],
        "Idempotency-Key": key,
    }
    body = {"concurrency": 4, "items": [e["item"] for e in events]}
    try:
        r = requests.post(API, json=body, headers=headers, timeout=30)
    except requests.RequestException:
        return "nack"
    if r.status_code == 202:
        return "ack"
    if r.status_code in (429, 503):
        return "nack"
    return "park"  # 400, 403, 409: retrying cannot fix these

When does redelivery still create duplicates?

The idempotency guard compares the whole payload. If K2 redelivers the same events but your consumer happens to slice them into a different batch the second time, the key changes and so does the item list, and Sume sees a brand new queue. Rows that already ran will run again, and each one is billed.

There are two fixes. Store a mapping of event id to queue id the moment you get a 202 and skip events already in it. Or submit one run per event with POST /v1/formats/{handle}/{slug}/runs and an Idempotency-Key equal to the event id, which makes every row its own guard no matter how batches are cut. That costs one write request per row instead of one per 100, which still sits well inside the write budgets listed on the errors page.

What about the results?

The queue has no webhook; communication.webhook_url is set per item, so put your receiver URL on each item. Poll GET /v1/format-run-queues/{id} for counts, and remember that a queue marked completed only means every item is terminal. Branch on counts.failed. K2's beta limits, 10 GB stored and 30 MB per second produced per stream, are far above what a 100-row batch of request bodies needs, but they are beta numbers from the announcement, so read the page again before you depend on them.

The announcement also lists anticipated prices after the beta, and says nothing is billed during it. Treat those as Cloudflare's to confirm, not as a quote.