Omni Flash image-to-video: frame_images or input_references?

Use frame_images with frame_type: "first_frame" when you want your photo to be the first frame of the clip. Use input_references when you want the photo to guide the look of the whole clip without being pinned as frame one. On Sume's /v1/videos, frame_images means image-to-video and input_references means reference-to-video. If you send both, frame_images wins.

Which field selects which mode

The docs call this out because of a past bug (#2044): a single bare image was ambiguous. On /v1/videos that ambiguous field does not exist. A single reference image goes in input_references and conditions the full clip, and a pinned opening frame goes in frame_images. On the legacy /v1/video-router/generate, the fields are image_url and reference_image_urls instead, and there a single reference image with no frame is also reference-to-video.

Pinned first frame

Here is the pinned version for Omni. For the reference version, replace the frame_images array with input_references holding an image_url entry, and expect the first frame not to match your photo.

{
  "model": "gemini-omni-flash-1.1",
  "prompt": "The camera pushes in slowly as steam rises from the cup",
  "duration": 5,
  "resolution": "720p",
  "aspect_ratio": "16:9",
  "frame_images": [
    {"type": "image_url", "image_url": {"url": "https://example.com/cup.jpg"}, "frame_type": "first_frame"}
  ]
}

Choosing

The choice shows in the result. If a viewer must see the exact product photo in the first second, pin it. If the photo only describes a character or a style and the scene should differ, use it as a reference. Omni accepts 16:9 or 9:16 only, so crop the photo to one of those first.