Sume 429 retry with a deadline: give up when retry-after is too long

A retry loop that always honors retry-after has a hidden cost. The wait on a Sume 429 can be as long as the rest of the 60-second window, and for queue_full the server default is 30 seconds. If a user is waiting on an HTTP response of yours, a 50-second sleep is not a retry, it is a hang. Give the loop a deadline.

The rule

Keep a budget in milliseconds for the whole call. After each 429, read the wait from retry-after. If the wait is larger than what is left of the budget, stop and throw the original error, so the caller can queue the work instead. Otherwise sleep, with a little jitter, and try again.

Which errors to retry

Wrapper

The wrapper retries only a 429 and reuses the same Idempotency-Key for a POST, so a retry never creates a second job. It throws the last response body when the budget runs out.

export async function fetchWithBudget(url: string, init: RequestInit, budgetMs = 60_000): Promise<Response> {
  const stop = Date.now() + budgetMs;
  for (;;) {
    const res = await fetch(url, init);
    if (res.status !== 429) return res;
    const body = await res.clone().json().catch(() => null);
    const secs = Number(res.headers.get("retry-after") ?? body?.error?.retry_after_seconds ?? 1);
    const waitMs = Math.max(1, secs) * 1000;
    const jitter = Math.random() * waitMs * 0.2;
    if (Date.now() + waitMs + jitter > stop) {
      throw new Error(`rate limited, retry-after ${waitMs / 1000}s exceeds the remaining budget: ${await res.text()}`);
    }
    await new Promise((r) => setTimeout(r, waitMs + jitter));
  }
}

// usage: await fetchWithBudget(url, { method: "POST", headers: { "x-api-key": key, "idempotency-key": id }, body })

What the caller does

On the thrown error, store the job request in your own queue with its idempotency key, and return 202 Accepted to your user. A worker can retry later with a longer budget. Because the key is fixed, the retry is safe even if the first attempt reached the API.

Choosing the budget

The right budget depends on who is waiting. A request that a person is waiting on can afford 5 to 10 seconds, which covers short waits but gives up on a long one. A background worker can take 120 seconds, which is two full windows. Since retry-after is at most the remainder of the 60-second window for a rate_limited response, a budget of 60 seconds or more always lets one retry through.

Test the wrapper against a local server that returns 429 once and then 200. Also test a server that always returns 429 with a 30-second delay and a 10-second budget, which must throw at once instead of sleeping.