Bulk queue says completed: it does not mean every episode rendered

No. A queue's status completed means every item has reached a terminal state, and terminal includes failed and canceled. Before you publish a season, read counts.failed and counts.canceled on GET /v1/format-run-queues/{id} and branch on them, not on the status alone.

What the status means

The reason the status is built this way is that a queue is a window over many independent runs. Each child can finish, fail or be canceled on its own, and the queue cannot know which outcome you consider acceptable. A season where episode 6 failed is still a finished queue; it is simply not a finished season.

The docs also note that the queue holds no output of its own. Results live on the child runs, so after the counts look right you still fetch each episode from its own run receipt.

The check before publishing

Treat a season as ready only when counts.failed and counts.canceled are both zero. If either is higher, list the items, find which episode numbers they were, and resubmit those. The same rule applies to a partly cancelled queue after someone stops an episode on purpose.

import os, requests

r = requests.get(
    f"https://api.sume.com/v1/format-run-queues/{os.environ['QUEUE_ID']}",
    headers={"Authorization": f"Bearer {os.environ['SUME_API_KEY']}"},
    timeout=30,
)
r.raise_for_status()
q = r.json()
c = q.get("counts", {})
bad = c.get("failed", 0) + c.get("canceled", 0)
if q.get("status") == "completed" and bad == 0:
    print("season ready")
else:
    print("not ready:", q.get("status"), "problem items:", bad)

Why webhooks do not replace this

A queue sends nothing when it finishes. Each item run can send its own webhook, and a canceled run never delivers one, so a handler that only counts deliveries will wait for episodes that will never arrive. Poll the queue for the overall picture, and use item webhooks to start downstream work for the episodes that did succeed.

Out of order is normal

With concurrency above 1, episode 7 can finish before episode 3. Keep the episode number in each item's input so you can sort the finished items afterwards; our bulk-queue ordering guide shows the publish step. For retries, resubmit only the failed items and give them new idempotency keys so a replay does not return the old queue; see retry only failed episodes.

A practical publish gate has three parts. First, the queue is completed. Second, counts.failed and counts.canceled are zero, or you can name each exception. Third, the number of finished children equals the number of episodes you meant to make. Only then should your scheduler flip the playlist to public, and a series that goes out one episode a week can instead release each episode as its own child finishes.