Remix a reference ad in two calls: ingest, then a sume-recreate run

To remix a reference ad with your product, make two calls. First, POST /v1/reference-ingest reads the reference clip into a manifest of shots, on-screen text, and audio facts. Second, POST /v1/formats/sume/sume-recreate/runs starts the Format with your product image and a spend cap. The ingest is optional but gives you facts to write the instruction against.

Call one: read the reference

Reference ingest takes one media.sume.com clip your workspace owns, up to 300 seconds, and returns a ReferenceVideoManifest. It lists frame-exact shots[], OCR text_tracks[], audio facts, visual boundaries[], and one labeled strip. It never re-encodes the source. Sume lists it only where the feature is enabled (development is on, production is opt-in), so check the dev host first.

The default mode is sync: the call waits up to 30 seconds and returns 200 with the manifest, or 202 with a queued job. Billing is by Modal compute, plus per-minute speech-to-text when the track has speech and speech.allow_billed_stt is on, which is the default for reference_remix.

Call two: run the Format

sume-recreate is one of the 27 slugs Sume lists at the reserved sume handle. Any key with formats:write can call it, and the run and its spend belong to your key. Read the Format first with GET /v1/formats/sume/sume-recreate for its io profile.

Send the product as an attachment, the reference in input, and your constraints in instruction. Name a cap in generation_spend_cap_usd. Without one, the run inherits the Format's cap, which is $400 when none was set, and the platform maximum is $500.

curl -sS -X POST "https://api.sume.com/v1/formats/sume/sume-recreate/runs" \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: serum-remix-v1" \
  -d '{
    "instruction": "Keep the reference scene order and caption rhythm. 9:16, new presenter.",
    "input": { "reference_url": "https://media.sume.com/artifacts/artf_ref/ad.mp4" },
    "attachments": [{ "type": "input_image", "image_url": "https://example.com/serum.png" }],
    "generation_spend_cap_usd": 25
  }'

Use the manifest in the instruction

The manifest gives you numbers to reuse. If audio.silent is true, tell the run to plan new music, because the source audio is unusable. If the shots average two seconds, say so. If OCR finds an offer line, say whether you keep its wording. The docs call the ingest a deterministic preflight before an agent plans a remix.

Media URLs inside input share the run's attachment budget: 30 in total, at most 30 images, 10 videos, and 10 audio. A reference clip counts as one video.

What to cap and what to retry

Set the cap to what one finished remix is worth to you, not to the $500 ceiling. The docs say Sume rejects a cap of 0 and a cap above 500 with 400, and null means the platform maximum. A run spends against its own cap, and the receipt shows both the cap and the spend so far.

If one scene is wrong, do not start over. Send previous_run_id with an instruction to redo that scene only, with its own cap and a new idempotency key. The continuation is a new run with its own receipt, and the original never changes.

Read the result

The create returns a receipt right away with status_url and result_url. Poll, or set communication.webhook_url for one signed terminal POST. A failed run still lists the media it made in artifacts[]. Check usage.billable_amount_usd_micros against the cap on the receipt.