Bulk run says completed: re-queue only the failed holiday SKUs

When a Sume bulk-run queue reports status: completed, it only means every item is terminal. Some of them can have failed or canceled. Read counts.failed and counts.canceled, collect those item index values, and submit a new bulk run containing only those items under a new Idempotency-Key. The items that finished are not run again.

This is the cleanest way to finish a seasonal catalog, because a queue holds up to 100 items and a single bad row should not make you re-pay for the other 99.

What does a bulk queue actually do?

A bulk request is a server-side queue of ordinary Format runs, per the Bulk runs page. POST /v1/formats/{handle}/{slug}/bulk-runs takes a concurrency from 1 to 16 and 1 to 100 items, each shaped like a single run body. The 202 receipt shows the first concurrency items already running and the rest queued.

The queue has no webhook of its own; communication.webhook_url is set per item. For queue-level progress you poll GET /v1/format-run-queues/{id}, which returns the same object as create, with counts and one row per item.

How do you find which items failed?

Each queue item has index, status, run_id and error. An item can end failed in two ways. If the child run failed, run_id is set and error.code is format_run_failed; read the run receipt at GET /v1/format-runs/{run_id} for the reason, because the queue item does not carry it. If the child could not start, run_id is null and error holds the create failure such as format_run_failed_to_start, and the rest of the queue keeps going.

A canceled item means a child run was canceled with POST /v1/format-runs/{run_id}/cancel. Decide per case whether to re-queue it.

What does the retry loop look like?

The script below submits eight items, polls the queue every 20 seconds until it is completed, then submits only the failed items as a second queue. The first call uses one idempotency key and the retry uses another, which matters: replaying a spent key returns 202 with the old queue, and the same key with a different payload returns 409 idempotency_conflict.

It reads the key from SUME_API_KEY, which needs the formats:write and formats:read scopes. Service-account keys cannot create Format runs or bulk queues, so use a normal key.

import json, os, time, urllib.request

KEY = os.environ["SUME_API_KEY"]
API = "https://api.sume.com/v1"

def call(method, path, body=None, idem=None):
    h = {"Authorization": f"Bearer {KEY}", "Content-Type": "application/json"}
    if idem:
        h["Idempotency-Key"] = idem
    data = json.dumps(body).encode() if body else None
    req = urllib.request.Request(API + path, data, h, method=method)
    with urllib.request.urlopen(req) as r:
        return json.load(r)["data"]

items = [{"instruction": f"Holiday clip for SKU {n}"} for n in range(1, 9)]
path = "/formats/sume/sume-close-camera-ugc/bulk-runs"
q = call("POST", path, {"concurrency": 3, "items": items}, "bf-batch-1")
while q["status"] != "completed":
    time.sleep(20)
    q = call("GET", "/format-run-queues/" + q["id"])
failed = [items[i["index"]] for i in q["items"] if i["status"] == "failed"]
print(q["counts"], "re-queue:", len(failed))
if failed:
    call("POST", path, {"concurrency": 3, "items": failed}, "bf-batch-1-retry-1")

What about cost and limits while retrying?

Every child is an ordinary Format run, so it passes ordinary admission: wallet, workspace generation concurrency and spend caps. Set generation_spend_cap_usd on each item to bound one run; the platform maximum is $500 and 0 is rejected.

The retry loop is not magic. If the failure is a bad input, such as an unreachable image URL, it will fail the same way again, so fix the row before you re-queue it. Sume does not expose a list-queues or cancel-queue endpoint, so keep the queue id from the first 202 in your own records.