Product hero video: pin the first frame to your own packshot

To make a product video that opens on your own packshot, send the image in frame_images with frame_type: "first_frame" on POST /v1/videos. The API reads that as image-to-video, so the clip's first frame is your photo and the model animates from there. With model: "sume/auto" the request is checked against a fixed envelope: 3 to 10 seconds, 360p to 4K, 16:9 or 9:16, with native audio always on. Sume does not say which model family serves an auto job, and the poll response echoes sume/auto.

Source: Video generation: the /v1/videos API. This is the stable path for a hero clip on a product page.

Why the first frame matters

On a product page the clip's poster image is often its first frame. If a model regenerates your product from a reference, the poster is a lookalike. If you pin the first frame, the poster is your photograph and the motion starts from something the customer already knows. That also means the packshot has to work as a video opening: a clean, well-lit, centred product on a simple surface, at the aspect ratio you ask for.

Request details

Send frame_images as a list: one item, type: "image_url", the URL, and frame_type: "first_frame". Do not send generate_audio: false on sume/auto; the Omni envelope has audio always on and rejects the value, so leave the field out or send true. The default duration is 8 seconds; set 6 if you want a tighter loop. The model echoes sume/auto in the response, and billing uses the resolved family because sume/auto has no price of its own.

Describe only the motion in the prompt: a slow push-in, the lid lifting, steam, a hand entering the frame. The image already says what the product is.

import json, os, time, urllib.request

H = {"Authorization": "Bearer " + os.environ["SUME_API_KEY"],
     "Content-Type": "application/json"}

def call(method, url, body=None):
    data = json.dumps(body).encode() if body else None
    req = urllib.request.Request(url, data=data, method=method, headers=H)
    with urllib.request.urlopen(req) as r:
        return json.load(r)

job = call("POST", "https://api.sume.com/v1/videos", {
    "model": "sume/auto",
    "prompt": "Slow push-in, the lid lifts, a thin curl of steam rises",
    "frame_images": [{
        "type": "image_url",
        "image_url": {"url": "https://media.sume.com/img/demo/kettle-packshot.png"},
        "frame_type": "first_frame",
    }],
    "aspect_ratio": "9:16",
    "duration": 6,
})
while job["status"] not in ("completed", "failed", "cancelled"):
    time.sleep(5)
    job = call("GET", "https://api.sume.com/v1/videos/" + job["id"])
print(job["status"])

What to check in the result

Compare the first frame of the output with the packshot: it should match, and everything after it is the model's work. Then check the last frame, because drift accumulates, and the label or logo may change over six seconds. If it does, shorten the clip or pin a last frame too, since frame_type accepts last_frame as well on models that list it.

Fetching the file and handling errors

When the job is completed, the poll response carries unsigned_urls; the first entry is GET /v1/videos/{job_id}/content?index=0, which redirects to the artifact. If you ask for content too early you get 409 job_not_completed, which is retryable; after a terminal failure you get 409 job_failed, which is not, so do not loop on it. The same job is also readable at GET /v1/jobs/{id}/status and /result in the normal Sume envelope.

Two failures are worth handling by name. A 400 unsupported_capability means a value is outside the envelope in the table, for example generate_audio: false or a 12-second duration. A failed job whose error says it could not download an input media URL means the packshot URL is not publicly reachable; use a public HTTPS URL and retry. Send an Idempotency-Key header on the submit so a retry after a network error returns the original job instead of a second charge. Sume reserves the cost when the job is admitted and the usage object reports the billable amount.

Using it across a catalogue

Make the prompt a template with the motion verbs and keep the packshot as the variable. The same structure across products means a bad result is easy to diagnose: it is the image, not the prompt. Keep the job id next to the SKU and read results from the job endpoints so a restart can resume.