Check an Omni edit kept the rest of the clip: video-frames pairs

To check that a Gemini Omni Flash edit kept everything else the same, extract stills from the source and from the edited clip at the same timestamps with POST /v1/video-frames, then compare each pair. The edit prompt asks the model to keep the rest of the scene, but the clip is regenerated, so only a side-by-side shows what moved.

Submit both extracts

Video frames takes one media.sume.com clip and either at[] (1 to 24 times, each from 0 up to but not including the duration) or fps. A submit always returns 202 with a request_id. Send an Idempotency-Key on REST so a retry does not queue a second extract. The script reads the two clip URLs from the environment and prints one request id for each.

import asyncio, json, os, urllib.request

def call(method, path, body=None, key=None):
    headers = {"Authorization": "Bearer " + os.environ["SUME_API_KEY"]}
    data = None
    if body is not None:
        data = json.dumps(body).encode()
        headers["Content-Type"] = "application/json"
        headers["Idempotency-Key"] = key
    req = urllib.request.Request("https://api.sume.com" + path, data=data, headers=headers, method=method)
    with urllib.request.urlopen(req) as resp:
        return json.load(resp)

async def main():
    times = [0.5, 2.5, 4.5]
    for name, url in {
        "source": os.environ["SOURCE_URL"],
        "edit": os.environ["EDIT_URL"],
    }.items():
        out = await asyncio.to_thread(
            call, "POST", "/v1/video-frames",
            {"video_url": url, "at": times}, "frames-" + name + "-001")
        print(name, out["request_id"])

asyncio.run(main())

Read the result

Poll GET /v1/video-frames/{request_id} until resource_status is ready. Each entry in frames has t, url, width and height. If a single instant fails, that frame has a null url, and the job still succeeds, so check for nulls before comparing. Add "format": "png" when you want lossless stills, or max_edge to clamp the long edge between 16 and 2160 pixels.

What to compare

Put the pair side by side and look in a fixed order.

Automate the verdict, not the judgment

You can script a rough check once you have the stills: compare image dimensions (the width and height in each frame entry should match across the pair) and flag any pair with a null url. Beyond that, a person needs to look. A pixel diff will flag a legitimate edit as a difference and a subtle face change as noise, so use it to choose which pairs to open first, not to approve a clip.

Keep the stills with the job record. When a client says "it looked different last time", the pair from the day of the edit settles it.

Choose times that matter

Pick the start, a moment in the middle where the edit is most visible, and a point just before the end. With a clip of 6 seconds, [0.5, 3, 5.5] is enough; with a 30-second edit, use a longer list up to the 24-frame limit. Avoid exactly 0 and the final frame, since the last time must be less than the duration.

For a whole-clip read in one call, video inspect returns a probe and eight stills by default. Video frames is the one to use when you need the same timestamps on both clips.