Test a Sume webhook receiver with signed fixtures, no paid job needed

You can test a Sume webhook receiver without spending anything, because the signature is only HMAC-SHA256 over <timestamp>.<raw_body> with a sume-v1= prefix. Pick a throwaway secret, sign a fixture body with it, and post the result to your route. Six cases cover what goes wrong in practice, and the script below checks all six against a reference verifier before you point it at your own.

The six fixtures

Each fixture isolates one rule from the Sume docs. If your receiver passes all six, its verification matches the documented contract. If it fails one, the table says which rule you broke.

The reference verifier and the checks

The script signs a body, then asserts the first five cases. It uses a fixed timestamp and passes now explicitly, so it runs the same every time and needs no network. The sixth case is a property of your router, not the verifier, so it belongs in your own tests.

import hashlib, hmac, json, time
SECRET = "whsec_test_not_a_real_secret"
def sign(secret, ts, raw):
    return "sume-v1=" + hmac.new(secret.encode(), f"{ts}.".encode() + raw, hashlib.sha256).hexdigest()
def verify(secret, ts, header, raw, now=None, tolerance=300):
    if not secret:
        return False
    try:
        age = abs((now or time.time()) - int(ts))
    except ValueError:
        return False
    if age > tolerance:
        return False
    want = sign(secret, ts, raw)
    return any(hmac.compare_digest(p.strip(), want) for p in header.split(","))
body = json.dumps({"event": "job.completed", "request_id": "job_x", "job_id": "job_x",
                   "status": "OK", "payload": {"artifacts": []}}).encode()
ts = 1780000000
good = sign(SECRET, ts, body)
old = sign("whsec_previous", ts, body)
assert verify(SECRET, ts, good, body, now=ts + 10)
assert verify(SECRET, ts, f"{good},{old}", body, now=ts + 10)   # rotation window
assert verify(SECRET, ts, f"{old},{good}", body, now=ts + 10)
assert not verify(SECRET, ts, good, body + b" ", now=ts + 10)   # reserialized body
assert not verify(SECRET, ts, good, body, now=ts + 301)         # outside the window
assert not verify("", ts, good, body, now=ts + 10)              # empty secret
print(good)
print("all fixtures pass")

Run the same fixtures against your route

Reuse sign to generate the headers, then post to your handler with x-sume-webhook-timestamp and x-sume-webhook-signature set. A good fixture should return 2xx, and each of the three rejection cases should return 401. Post the same good fixture twice and check that your own storage holds one row, because a real delivery is at-least-once and the second copy must be a no-op.

Add two payload shapes to your fixtures from the docs: a job.completed body with payload.artifacts, and a run receipt where payload is null and error.code is payload_too_large. The second one is a signed, valid delivery for a run that finished, so it must not be treated as a failure of the run.

Keep the fixtures in the repository as files, not as strings built inside each test. A body that is checked in as bytes, with its expected signature for the throwaway secret, is a regression test for the exact thing that breaks first when someone changes a middleware: the bytes your handler reads. If a framework upgrade changes how the body is parsed, the checked-in fixture fails the same day.

Name each fixture for the rule it protects, such as reserialized-body-rejected, so that a red test tells the next engineer which line of the contract just changed.

What a fixture cannot prove

Fixtures prove your verifier, not your transport. They cannot show that your platform hands the handler the original bytes, because a framework that parses and rewrites JSON will pass a unit test and still fail every real delivery. Test that last part once with a real signed delivery.

Sume's Send test action posts a dummy signed webhook.test event to a URL you type, and POST /v1/webhooks/test-deliveries does the same with a key that has account:write. It is not a replay of a real job. To replay an actual terminal event, use POST /v1/jobs/{id}/webhook/redeliver, or the Format run equivalent, which sign again with a fresh timestamp and the same secret.

One more habit is worth keeping. Never put the production secret into a fixture or a test environment variable that gets logged. The throwaway secret exists so that a leaked test log costs nothing, and the verifier under test should refuse an empty secret, which catches the case where the environment variable was never set. Add a seventh test for that case if your deployment reads the secret from the environment: start the app with the variable unset and check that every delivery gets a rejection and not a pass.