Bulk queue, empty wallet: failed items, fresh key, check balance

What happens

A Sume bulk run returns 202 once the queue is accepted. After that, each child run still has to pass admission, including the wallet check. If the wallet runs short mid-run, those children become failed items that carry the create-run error, and the queue's window refills with the next items. The queue ends completed once every item is terminal, and counts.failed tells you how many fell out.

Nothing about this is silent if you read the counts. A queue marked completed with failed items is an expected outcome, not a contradiction.

Read, top up, resubmit

The table gives the sequence.

The fresh key rule

Reusing the original Idempotency-Key for the resubmit is the common mistake. A replayed key returns 202 with the old queue, so you get no new work and may think the retry succeeded. Build the new key from the batch and the retry round, for example launch-batch-7-retry-1.

Resubmit only the failed items. Items that completed already produced their output and their charge.

curl -sS https://api.sume.com/v1/balance \
  -H "Authorization: Bearer $SUME_API_KEY" | jq .

Avoid it next time

Compare the worst-case ceiling to the balance before you start: items times the per-item cap. For 100 items at a $20 cap that is $2,000, an upper bound, not a forecast. Top up to the level you are comfortable with, or lower generation_spend_cap_usd per item.

Failed items have a generic format_run_failed on the queue, so read each child receipt for the true reason before you retry. A retry for an error that is not about funds will fail again.

• Check GET /v1/balance before big queues.
• Keep concurrency modest, so a short wallet stops fewer runs in flight.
• Alert on counts.failed greater than zero.

Worked example

Say you queued 150 rows as two queues, and the wallet ran dry after 120 children finished. The first queue completes with counts.failed at zero. The second shows 30 failed, each with the create-run error. Check the balance, top up, build a new batch from those 30 rows and send it with a key like spring-retry-1. You pay for 30 runs, not 150.

• Never retry the whole queue.
• Record which rows came from which attempt.
• Treat these as working notes you can adapt: the figures are from the Sume docs read on 2026-10-08, and the arithmetic is yours to rerun with your own numbers.

Make it routine

Add this check to your runbook and run it on a schedule, not only after an incident. The cost is a few read requests, and the rate limits are far above what it needs: even the Free plan allows 120 writes and 4,800 reads per minute. Keep the output with the date, so you can show later what the system looked like when a question came up.