terminal, result_ready or status: which check ends a Sume poll loop?

In a Sume poll loop, terminal decides when to stop polling, result_ready decides when it is safe to call /result, and sume_status decides what the job ended as (completed, failed, canceled). The status field is a queue-shaped copy (COMPLETED, FAILED, CANCELED) that always agrees with it; do not mix the two. Using one field for all three is the common bug: stopping on result_ready never ends for a failed job, and stopping on sume_status == "completed" skips canceled and failed.

The jobs docs say to poll status_url until terminal is true, then read result_url when result_ready is true.

Three fields, three questions

Statuses come from the jobs and results page; terminal ones include completed, failed and canceled.

Why a failed job is terminal but not ready

A failed or canceled job is terminal, so the loop must stop, but result_ready is false and /result would answer 409 job_not_completed. That is why failures are read from the job record. Also, the loop must have a client deadline: a client-side timeout does not cancel the job, which keeps running and billing, so either store the id and come back or call cancel while it is still queued.

Also respect next_poll_after_seconds over a fixed sleep. Reads have a budget, and polling faster than asked adds load and nothing else.

The loop

Standard library only; set SUME_API_KEY. It returns the job record on a failure and the result on success.

import json, os, time, urllib.request

BASE = "https://api.sume.com/v1"
HEAD = {"Authorization": "Bearer " + os.environ["SUME_API_KEY"]}


def get(path):
    req = urllib.request.Request(BASE + path, headers=HEAD)
    with urllib.request.urlopen(req, timeout=30) as resp:
        body = json.load(resp)
    return body.get("data", body)


def finish(job_id, deadline_s=1200):
    end = time.time() + deadline_s
    while time.time() < end:
        st = get(f"/jobs/{job_id}/status")
        if st.get("terminal"):
            if st.get("result_ready"):
                return "ok", get(f"/jobs/{job_id}/result")
            return st.get("sume_status"), get(f"/jobs/{job_id}")
        time.sleep(float(st.get("next_poll_after_seconds") or 3))
    return "deadline", {"job_id": job_id}

Edge cases

Treat an unknown status as non-terminal unless terminal says otherwise, so a new status value does not end your loop early. Keep the deadline outside the loop's polling cost, and log the job id on every path, including the deadline.

Testing the loop

Feed the loop three canned payloads: one in flight, one completed, one failed. Assert that the first sleeps for the requested seconds, the second fetches the result, and the third returns the job record without calling /result. Add a fourth with an unknown status and terminal: false, and assert that it keeps waiting.

For several jobs, run one loop per job id with a shared pause based on the smallest next_poll_after_seconds, and drop each id as it reaches terminal. The shared loop keeps reads far below the budget, which is forty times the write limit, even with a full queue of accepted jobs.

These four cases catch the loop bugs that cost real money: an early exit that abandons a running job, and a spin that polls faster than asked.

• Mock get and time.sleep.
• Assert on call counts, not only on return values.
• Keep the deadline injectable so a test does not wait twenty minutes.