arq worker that polls an AI video job with defer_by in Python

arq is an asyncio job queue on Redis, and it fits video polling because a task can enqueue its own successor with _defer_by. The task reads Sume's GET /v1/jobs/{id}/status, returns when terminal is true, and otherwise enqueues the same task again after next_poll_after_seconds (Sume jobs guide, read 2026-10-06). One worker process can track hundreds of renders because each task is a short HTTP read.

The submit step below uses seedance-2-mini at 720p for 8 seconds. At Sume's rate that is $1.512 (the 10 second price of $1.89 scaled to 8 seconds).

What are the submit and poll tasks?

ctx['redis'] is the arq pool passed to every task. The Idempotency-Key means an arq retry of submit returns the same job. Start the worker with arq worker.WorkerSettings.

import os
import httpx
from arq.connections import RedisSettings

API = "https://api.sume.com"
H = {"Authorization": f"Bearer {os.environ['SUME_API_KEY']}"}

async def submit(ctx, prompt: str, key: str):
    async with httpx.AsyncClient(timeout=30) as c:
        r = await c.post(f"{API}/v1/videos", headers={**H, "Idempotency-Key": key},
                         json={"model": "seedance-2-mini", "prompt": prompt,
                               "duration": 8, "resolution": "720p"})
        r.raise_for_status()
    await ctx["redis"].enqueue_job("poll", r.json()["id"], 0, _defer_by=15)

async def poll(ctx, job_id: str, n: int):
    async with httpx.AsyncClient(timeout=30) as c:
        r = await c.get(f"{API}/v1/jobs/{job_id}/status", headers=H)
        r.raise_for_status()
    s = r.json()
    if s["terminal"]:
        return s["sume_status"]
    if n < 120:
        wait = s.get("next_poll_after_seconds") or 15
        await ctx["redis"].enqueue_job("poll", job_id, n + 1, _defer_by=wait)

class WorkerSettings:
    functions = [submit, poll]
    redis_settings = RedisSettings()

Why does a queued job need a longer delay?

A job that is accepted but not yet running is queued, and the status response suggests a delay of up to 30 seconds for it. Sume runs a limited number of jobs per workspace at once (Pro: 4 processing and 20 queued, per the generation admission guide), so submitting 30 clips at once leaves most waiting. Honoring the suggested delay keeps the read count low during that wait.

Does a retry of the poller bill a second clip?

No, as long as the poller only reads. Keep the paid POST /v1/videos in a separate step with an Idempotency-Key you own, and let the poller use GET /v1/jobs/{id}/status. A client-side give-up never cancels the render; Sume's jobs guide says the job keeps running, so store the job id and read it again later instead of submitting a second time.