Windmill sync returns 200 on error: check Sume's failed flag too

The answer

Windmill's webhook page states that in synchronous mode, if the script returns an error, the default behavior is a 200 status code with the error as a JSON object, and that you can customise the HTTP status. A caller that treats 200 as success will mistake a failed Windmill run for a good one.

Sume's API gives you a field to avoid that trap on its side: a sync or subscribe response carries a sync object with completed, succeeded, failed, canceled, terminal and timed_out. Branch on those, not on the status line.

Where a 200 can hide a failure

There are two layers. The Windmill layer reports a script error as JSON with 200. The Sume layer reports a failed generation as a job whose status is failed, which is a legitimate response to a status read. Neither is an HTTP error.

A check that covers both

In the script, call Sume, then examine the parsed body. If data.sync.failed is true, raise an error so Windmill records a failure; if timed_out is true, return the job id for a later status read. If it succeeded, return the result reference. Raising on failure makes Windmill's own error handling and your alerts work as designed.

After a raise, check the job events through events_url for a sanitized reason before deciding whether a new submit is warranted. A failed job may have been billed or not; the usage object on the response shows the estimate, and the errors page lists the HTTP errors that stop a request before it becomes a job.

Customise the Windmill status

Because Windmill allows a custom HTTP status, set it from the script when you want callers to see a 5xx or 4xx on failure. Pick codes consistent with Sume's own: 402 for credits, 429 for capacity, 409 for not yet complete.

Testing the failure path

Write one test that makes the Windmill script fail on purpose, for example by passing an invalid model id, and assert that your caller treats the result as an error and not as data. Then write a second test that feeds the caller a Sume status body with a failed job and assert the same. Both tests are short, and they stop the most common bug in these integrations: a success path that quietly handles a failure body.

Log the request id from the response when a job fails. It ties your log line to that call without storing any prompt text.