A kill switch for paid Sume submits: stop new jobs, cancel queued

A kill switch for Sume spend has two parts: a flag your code reads before every paid submit, and a sweep that cancels the jobs that have not started yet. Cancellation only works before generation starts, so the sweep cannot recover everything; plan for the rest.

What cancel can and cannot do

POST /v1/jobs/:id/cancel succeeds only before generation work starts. Once it has started, the API returns 409 job_generation_already_started with details.cancelable: false, and the job runs to completion. Cancelling a job that is already canceled is idempotent, so the sweep can be re-run safely.

The sweep

Keep the job ids you submit in your own store, then walk them on shutdown of a feature. The reservation for a canceled job is released; a job that already started finishes and captures its cost.

import asyncio, os
import httpx

async def cancel_all(job_ids: list[str]) -> None:
    headers = {"x-api-key": os.environ["SUME_API_KEY"]}
    async with httpx.AsyncClient(base_url="https://api.sume.com", headers=headers) as c:
        for job_id in job_ids:
            r = await c.post(f"/v1/jobs/{job_id}/cancel")
            code = r.json().get("error", {}).get("code") if r.status_code == 409 else None
            if code == "job_generation_already_started":
                print(job_id, "already running; it will finish and bill")
            elif r.is_success:
                print(job_id, "canceled")
            else:
                print(job_id, "needs a look", r.status_code)

async def main() -> None:
    await cancel_all(["job_123"])

asyncio.run(main())

The flag side

Check the flag at the one function that submits, not at each caller. When it is off, return a clear error to your own users instead of queueing work in memory, which would burst out when the flag flips back on.