Replace your mocked Sora client with a Sume contract test

Tests that mocked a Sora client now guard nothing. Write a small pytest contract test around your own video interface, plus one live smoke test against Sume.

5 min readSume
All posts

If your test suite mocked a Sora client, those tests now prove only that your mock still works. Replace them with two layers. First, a contract test against your own video interface using recorded Sume response shapes from the video generation docs: submit returns an id, polling_url and pending, and polling ends in completed with unsigned_urls or in failed. Second, one opt-in live smoke test that spends real money and runs only with a key set.

What each layer owns

Keep the layers distinct so a flaky network never fails the main suite.

Test layers for a video integration (read 2026-10-04)
LayerNetworkChecksWhen it runs
Unit with fixturesNoneRequest building, status mapping, error mappingEvery commit
Contract fixturesNoneDocs-shaped responses parse into your typesEvery commit
Live smokeSume APIOne short job completes and returns a URLNightly or manual

The fixtures to write

Copy the shapes from the docs, not from memory: the 202 submit response, a poll response mid-job, a completed poll with unsigned_urls and usage.cost, a failed poll with an error, and the error envelope for 402 and 429 from the errors guide. Name each fixture for the behavior it proves.

A pytest sketch

This test feeds a fake poll sequence into a status-mapping function you own. It is runnable as written once placed in a file.

def map_status(s):
    return {"pending": "queued", "in_progress": "running",
            "completed": "done", "failed": "failed",
            "cancelled": "failed", "canceled": "failed"}[s]

def test_poll_sequence_ends_done():
    seq = ["pending", "in_progress", "in_progress", "completed"]
    assert [map_status(s) for s in seq][-1] == "done"

def test_unknown_status_raises():
    import pytest
    with pytest.raises(KeyError):
        map_status("mystery")

The live smoke test

Gate it with an environment check so it skips without a key. Submit one short, low-resolution job with an Idempotency-Key, poll the job status with backoff, and assert you reached completed with a downloadable URL. Keep the prompt trivial: the test checks the plumbing, not the model quality. Expect it to take minutes and to cost real money, so run it nightly rather than per commit.

Sources

Related posts

More in Developers

All Developers posts

Written by Sume