Use a frame from your footage as a reference for AI video

Extract the frame with POST /v1/video-frames, naming the time in at[], then pass the returned image URL to a video model: as image_url for a start frame, or in reference_image_urls (up to 10) for a reference. Adobe's Premiere 26.5 notes say its Generative Media Tool lets editors use existing frames as visual references; on Sume you do the same in two API calls.

Details are from Video frames and Video Router, read 2026-09-30.

How do I pull the frame?

Video frames takes one media.sume.com clip and exactly one of at[] (1 to 24 times in seconds) or fps. Optional format is jpeg (default) or png, and max_edge is 16 to 2160; omit it to keep the source size. The call is always 202 async, and the call is unbilled. Each time must be within the clip, or the worker fails with frame_time_out_of_range.

curl -X POST https://api.sume.com/v1/video-frames \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: ref-frame-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "at": [4.2],
    "format": "png"
  }'

How do I hand the frame to a video model?

When resource_status is ready, frames[{t,url,width,height}] hold durable image URLs. On gemini-omni-flash-1.1 the request shape picks the capability, as the table shows.

Which frame should I choose?

Pick a frame where the subject is sharp and the framing is what you want the new clip to start from. Not only the last frame: at[] takes any time, so you can pull several candidates in one call and compare. For the last-frame case, see get the last frame of a video to chain clips.

What are the limits?

The source clip can be up to 300 seconds, at most 24 frames per call, and it must be a Sume-hosted artifact (import it first with POST /v1/media-imports). The video model's own limits apply to the new clip: gemini-omni-flash-1.1 is 3 to 10 seconds at 16:9 or 9:16. A reference guides the generation; it does not guarantee the new clip matches the original footage.