Sume bulk queue has no webhook: poll status_url, read counts.failed

When a Sume Format bulk queue shows status: completed, it means every item has finished, not that every item worked. Read counts.failed and the error on each item. The queue object also has no webhook: only individual items can register communication.webhook_url, so queue-level progress comes from polling the status_url, which is GET /v1/format-run-queues/{id} and needs the formats:read scope. These rules are in the bulk runs docs.

For a Black Friday overnight batch, that means a small watcher loop, and one extra check at the end.

What the receipt tells you

Create returns 202 with a frq_ id, the concurrency window, counts (total, queued, running, completed, failed, canceled), an items list in submitted order and a status_url. Each item has index, status, run_id and error.

A watcher that ends correctly

Poll with backoff, stop when status is completed, and then decide what to do about failures. A short shell loop is enough, and it reads the receipt URL from the create response, not a path you built.

QUEUE_URL="https://api.sume.com/v1/format-run-queues/frq_example"
while :; do
  Q=$(curl -sS "$QUEUE_URL" -H "Authorization: Bearer $SUME_API_KEY")
  S=$(echo "$Q" | jq -r '.data.status')
  [ "$S" = "completed" ] && break
  sleep 30
done
echo "$Q" | jq '{counts: .data.counts, failed: [.data.items[] | select(.status=="failed") | .index]}'

Reading the failures

An item that failed after a child run began has error.code format_run_failed; fetch GET /v1/format-runs/{run_id} for the full receipt and its own error. An item that failed before a child run started keeps its index, has run_id: null and carries the create failure, for example format_run_failed_to_start. The rest of the queue keeps going, so one bad SKU never stops the other 99.

Collect the failed indexes, fix the inputs and send them as a new, smaller queue under a new Idempotency-Key. Do not replay the original key, because the API then returns the old queue.

If your system truly needs push notification, register a webhook per item, and still poll the queue as a backup for the day an endpoint is down.

• Poll status_url; no queue webhook exists.
• completed is not all succeeded.
• Re-queue only the failed indexes with a new key.

Poll rhythm and scopes

Pick a polling interval that matches the work. A Format run is one sandbox and one agent turn, not a quick ffmpeg step, so polling a queue every 30 seconds is plenty and polling every second only adds load. Use a key with formats:read for the watcher and a separate key with formats:write for the submitter, if your security rules ask for least privilege. If a team workspace owns the Format, both keys must be issued in that workspace; service-account keys cannot create Format runs or bulk queues at all and fail with 403 insufficient_scope.

Log the queue id and the status_url the moment the create returns. That one line is what lets a restarted watcher pick up where the last one stopped, and the queue itself keeps running on the server while your laptop sleeps. This is the point of the design: you can leave the list to run overnight without driving the fan-out from your own machine.

In the morning, read the receipt, sum counts, and check the number against counts.total. The five counts should add up to the total once the queue is terminal, with no item still queued or running. If they do not, something is still in flight and the status would not yet be completed.