Caption a folder of videos in Python: thread pool, one key per file

Submit one POST /v1/video-captions per file, give each its own Idempotency-Key derived from the file and the style, and poll GET /v1/jobs/:id/status until terminal is true. A thread pool works well for that because every wait lives in your client, and a retry that reuses the key returns the original job instead of billing a second one. Each accepted standalone caption job for a clip up to 60 seconds is $0.20, so a duplicate is a real cost, not a rounding error.

The mechanics below come from Sume's Jobs and results, Generation admission and video captions docs.

What does one file's flow look like?

Submit returns 202 with a job envelope: request_id (the job id), status_url, result_url and next_poll_after_seconds. The default mode is async. Then poll status, honoring next_poll_after_seconds when present and backing off otherwise, and stop when terminal is true. A client timeout does not cancel the job; it keeps running and keeps billing, so store the job id and resume from status_url rather than resubmitting.

How should the idempotency key be built?

The docs say to reuse the same key only for the same operation and payload. So hash the video URL together with the style: rerunning the script after a crash sends the same key for the same file and gets the same job back, while a new style on the same file is a new operation with a new key. A random key per attempt defeats the purpose, since a retry would then look like a new request.

How many files can you submit at once?

Concurrency is a dispatch limit, not a submit limit: valid jobs beyond your workspace's concurrency are accepted as queued while queue capacity remains. The docs' table of default capacity by plan follows; prefer the effective generation_limits.concurrency_limit field over a static table.

What does the script look like?

The function below submits with a derived key, polls, and returns the job id and final status. It reads the API key from SUME_API_KEY and uses only the standard library. Point urls at public HTTPS videos.

import hashlib, json, os, time, urllib.request
from concurrent.futures import ThreadPoolExecutor

API = "https://api.sume.com"

def call(method, path, body=None, key=None):
    h = {"Authorization": "Bearer " + os.environ["SUME_API_KEY"],
         "Content-Type": "application/json"}
    if key:
        h["Idempotency-Key"] = key
    data = json.dumps(body).encode() if body else None
    req = urllib.request.Request(API + path, data, h, method=method)
    with urllib.request.urlopen(req, timeout=60) as r:
        return json.load(r)

def caption(url, style="slam"):
    key = "cap-" + hashlib.sha256(f"{url}|{style}".encode()).hexdigest()[:24]
    job = call("POST", "/v1/video-captions",
               {"video_url": url, "style": style}, key)
    while True:
        s = call("GET", f"/v1/jobs/{job['request_id']}/status")
        if s.get("terminal"):
            return job["request_id"], s.get("sume_status")
        time.sleep(s.get("next_poll_after_seconds") or 5)

urls = ["https://example.com/a.mp4", "https://example.com/b.mp4"]
with ThreadPoolExecutor(max_workers=4) as pool:
    for job_id, status in pool.map(caption, urls):
        print(job_id, status)

What happens on 429?

Two different 429s exist. rate_limited means too many requests in the current window; back off, use retry-after when present, and resend with the same idempotency key. queue_full means the workspace cannot accept another paid generation job until a queued or processing one finishes or is canceled, so pause submitting rather than hammering it. The script above lets urlopen raise on these, which is deliberate for a first run: wrap call in a retry that sleeps on retry-after once you know your limits. Do not retry an unsafe submit without a key.

Last, read results from GET /v1/jobs/:id/result once the job is completed; it answers 409 job_not_completed before that. A failed caption job carries a typed error such as caption_no_speech, so check sume_status before you assume a file exists.