Bulk queue concurrency 16 but fewer runs start: workspace concurrency

Concurrency 16 is a ceiling on the queue, not a promise. Workspace generation concurrency still applies to every child, so some start late or fail to start.

3 min readSume
All posts

Setting concurrency: 16 asks the queue to keep 16 children in flight. It does not raise your workspace's own generation concurrency, which the docs say still applies to each child.

What the docs say

  • concurrency is an integer from 1 to 16: the number of child runs the queue keeps in flight.
  • Children go through ordinary Format-run admission: wallet, workspace generation concurrency and spend caps.
  • If a child fails to start, that item becomes failed with a code such as format_run_failed_to_start and run_id: null, and the window refills from the rest.
  • Items always run with on_active_run: allow, so a skip or reject on an item will not serialize the batch.

What it means for you

Do not read a failed item as a bad input before looking at its error. A start failure that comes from admission is not the same as a run that failed. Requeue those items; re-send only the failed rows in a new queue with a new key.

def failed_rows(queue, ledger):
    return [ledger[it["index"]] for it in queue["items"]
            if it["status"] == "failed"]

Pick a window you can feed

Start lower than 16 for a first run, watch counts and the item errors, then raise it. A smaller window finishes a bit later but wastes less on failures to start.

Sources

Related posts

More in Formats

All Formats posts

Written by Sume