Assert the Sume usage ledger in CI: canceled job debits zero

Call GET /v1/usage?job_id=... after canceling a queued job and assert two fields of the summary: final is true and debited_usd_micros is 0. The Usage docs define final as true when no hold is open, and debited_usd_micros as what the wallet deducted, so together they say the reservation was released and nothing was spent.

That is a ledger-level test of your cancel path, and it catches the mistake that unit tests cannot: a cancel that your code sent but that arrived too late.

What the ledger statuses mean

The docs list three row statuses. A reserved row is an estimate held before provider execution, captured is billable usage after a successful completion, and refunded is a reservation released after a failure or a cancellation before capture. A refunded row keeps its hold amount in billable_amount_usd_micros, so you cannot read spend off that field alone; use the summary.

The docs also say never to sum the rows yourself. The summary folds every row the scope caused, and limit only caps the listed rows.

The test

Pass a job id that you submitted and that is still queued. The script cancels it, reads the usage for that job, and asserts. If generation already started, Sume answers 409 with details.cancelable: false, and the script exits with that body, which is the right failure: the job will run and be billed.

import json, os, sys, urllib.error, urllib.request

BASE = "https://api.sume.com"
HEAD = {"x-api-key": os.environ["SUME_API_KEY"], "Content-Type": "application/json"}


def call(method: str, path: str, body: dict | None = None) -> dict:
    data = json.dumps(body).encode() if body is not None else None
    req = urllib.request.Request(BASE + path, data, HEAD, method=method)
    try:
        with urllib.request.urlopen(req, timeout=30) as r:
            return json.load(r)
    except urllib.error.HTTPError as e:
        sys.exit(f"{method} {path} -> {e.code} {e.read().decode()}")


job_id = sys.argv[1]  # a job you submitted that is still queued
call("POST", f"/v1/jobs/{job_id}/cancel")
summary = call("GET", f"/v1/usage?job_id={job_id}&limit=20")["summary"]
assert summary["final"] is True, "a hold is still open"
assert summary["debited_usd_micros"] == 0, summary["debited_usd_micros"]
print("canceled, nothing debited, refunded:", summary["refunded_usd_micros"], "micros")

Making it reliable in CI

Submit a job you know will queue, for example by filling the concurrency slots in a dev workspace first, otherwise the job can start before your cancel lands. The admission docs explain that a job waits as queued while the workspace is at its concurrency limit.

Run this on the dev host with a development key, with a spend cap on the submit, and keep it out of pull request checks that run in parallel; a nightly schedule is enough to notice a regression in your cancel handling.

The money values are integers in millionths of a dollar, so compare them with == and never convert to floats in a test.