Unit test Sume webhook signature checks in pytest (Python)

To unit test webhook signature verification in Python, build the signature in the test with the same recipe the sender uses, then assert four things: a good delivery passes, a one-byte change to the body fails, a stale timestamp fails, and an empty secret raises instead of quietly accepting. For Sume the recipe is HMAC-SHA256 over <timestamp>.<raw_body>, sent as sume-v1=<hex> in x-sume-webhook-signature next to x-sume-webhook-timestamp.

Sume's docs describe a TypeScript SDK with verifyWebhook and list no Python package, so a Python receiver implements the scheme itself. That is a dozen lines, which is exactly why it deserves tests: the failures that hurt are silent ones, such as a verifier that returns true for everything or breaks on the day you rotate the secret.

What does the verifier have to get right?

Sume's docs give the rules, and each one maps to a test case below. The tests need no network and no Sume key, because you are the signer.

The verifier to test

Header names are case-insensitive, so the function lowercases them. It returns False for a malformed delivery rather than raising, matching how Sume's own verifyWebhook behaves, and it checks every comma-separated entry without stopping at the first match.

# sume_verify.py
import hashlib
import hmac
import time


def verify(raw_body: bytes, headers: dict, secret: str, tolerance: int = 300) -> bool:
    if not secret:
        raise ValueError("refusing to verify with an empty secret")
    h = {k.lower(): v for k, v in headers.items()}
    try:
        ts = int(h["x-sume-webhook-timestamp"])
        header = h["x-sume-webhook-signature"]
    except (KeyError, ValueError):
        return False
    if tolerance and abs(time.time() - ts) > tolerance:
        return False
    msg = str(ts).encode() + b"." + raw_body
    expected = "sume-v1=" + hmac.new(secret.encode(), msg, hashlib.sha256).hexdigest()
    ok = False
    for entry in header.split(","):
        if hmac.compare_digest(entry.strip(), expected):
            ok = True
    return ok

The pytest file

Run it with pytest from the folder holding both files. The helper signs with any number of secrets at once, which is the same shape as the rotation header.

# test_sume_verify.py
import hashlib, hmac, time
import pytest
from sume_verify import verify

BODY = b'{"event":"job.completed","job_id":"job_1"}'

def sig(secret, ts):
    mac = hmac.new(secret.encode(), f"{ts}.".encode() + BODY, hashlib.sha256)
    return "sume-v1=" + mac.hexdigest()

def hdrs(*secrets, ts=None):
    ts = int(time.time()) if ts is None else ts
    return {"X-Sume-Webhook-Timestamp": str(ts),
            "X-Sume-Webhook-Signature": ",".join(sig(s, ts) for s in secrets)}

def test_valid_and_tampered():
    assert verify(BODY, hdrs("new"), "new")
    assert not verify(BODY + b" ", hdrs("new"), "new")

def test_rotation_header_accepts_either_secret():
    h = hdrs("new", "old")
    assert verify(BODY, h, "old") and verify(BODY, h, "new")

def test_stale_timestamp_and_empty_secret():
    assert not verify(BODY, hdrs("s", ts=int(time.time()) - 3600), "s")
    with pytest.raises(ValueError):
        verify(BODY, hdrs("x"), "")

After the check passes, what should the handler do?

Verification only says the delivery is genuine. The docs then ask for three more things, and each is testable without a network. Dedupe on the right key: job_id for job webhooks, and the envelope's request_id (equal to run_id) for run webhooks, because retries repeat the same value. Return a 2xx quickly after durably recording the event, since each attempt has a 10-second timeout and a slow endpoint burns the attempt and is retried. And treat an unknown event as 204 instead of a 500, so a new event type never becomes a retry storm.

Write those as tests too: post the same signed body twice and assert one row, post a webhook.test body and assert a 2xx with no side effects, and post a body whose event you have never seen. A receiver that passes all of these survives a rotation, a retry and a new event type, which are the three things that actually change in production.

Which cases do teams skip, and what do they cost?

The rotation case is the one most often missing. Sume signs with both secrets for 24 hours after you rotate, and a hand-rolled verifier that compares the header for equality fails every delivery in that window; the docs call this out and say to upgrade the receiver before you rotate. The raw-body case matters just as much: a framework that parses JSON first has already changed the bytes, so the signature can never match. Test your route with a body that has unusual key order or whitespace, since that is what reserialization breaks.

The empty-secret case guards a deployment mistake, not an attack. If SUME_COM_WEBHOOK_SIGNING_SECRET is unset and your code reads it with a default of an empty string, HMAC with an empty key still produces a valid-looking digest, and a verifier that does not refuse it will happily match a signature anyone can compute. Failing loudly at startup is cheaper than that.

Finish with one end-to-end check outside the unit tests: fire a real signed delivery at the deployed route (see the test delivery gate). Unit tests prove your logic; the live delivery proves the secret you deployed is the one Sume signs with.

• Pin the clock by passing explicit timestamps, as the helper does, rather than sleeping.
• Keep the signing helper in the test file only; production code should never be able to sign.
• Add a case per event type your router handles, since the signature is the same for job.* and format.run.terminal.