Reference-to-video API: guide a clip with images, video, and audio

To guide a video with reference media through the Sume API, send POST /v1/videos with an input_references array of image_url, video_url, or audio_url entries. The model uses references as style or content guidance, not as exact frames, and accepts only the types its catalog entry lists in supported_input_references.

The request shape comes from the Video generation docs. Per-model types and limits come from the catalog behind GET /v1/videos/models and the checks POST /v1/videos runs, read on 2026-09-26.

How do I send reference images, videos, or audio?

Each entry names its type and nests the URL under a key of the same name, such as { "type": "video_url", "video_url": { "url": "…" } }. This request extends the docs' reference-to-video example with a video reference next to the image:

• Reference URLs must be public HTTPS. Localhost, private-network, non-HTTPS, and signed or private URLs are rejected before submission.
• Leave frame_images out. If a request carries both, frame_images takes precedence, the request becomes image-to-video, and the references are not used. Frames are covered in Image-to-video API with first and last frames.

curl -X POST https://api.sume.com/v1/videos \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: r2v-001" \
  -d '{
    "model": "seedance-2",
    "prompt": "A colossal solar flare beside a planet",
    "input_references": [
      {
        "type": "image_url",
        "image_url": { "url": "https://example.com/style-ref.png" }
      },
      {
        "type": "video_url",
        "video_url": { "url": "https://example.com/motion-ref.mp4" }
      }
    ],
    "resolution": "1080p"
  }'

Which models accept image, video, or audio references?

Seedance 2.x, Wan 3.0, MiniMax H3, and MiniMax H3 Max accept all three types. Gemini Omni Flash 1.1 takes images and videos but not audio. kling-3 and grok-imagine-video-1.5 take no references.

What happens if I send a type or count the model does not take?

POST /v1/videos refuses the request with a 400. A type or per-type count the model does not take gets the error code unsupported_capability and a message that names the limit:

• A type the model does not list: “kling-3 does not support input_references type image_url.”
• Too many of one type: “wan-3.0 accepts at most 5 video input_references.”
• More than 12 references in total never reaches those checks: the request body itself allows at most 12 input_references, so a 13th fails as invalid_request.

Are there model-specific rules for references?

Two model families add rules of their own:

• Gemini Omni Flash 1.1: address reference media in the prompt as <IMAGE_REF_0> and <VIDEO_REF_0>, counted from 0 in list order. A reference video guides a new clip; to change an existing clip, its edit source is a separate video_url field, covered in Edit a video with a prompt.
• MiniMax H3 and H3 Max: pair audio with at least one image or video reference.

How do references work on the Video Router?

The legacy POST /v1/video-router/generate route still works unchanged and takes references as flat URL arrays: reference_image_urls, reference_video_urls, and reference_audio_urls. Its own checks differ in two places: Seedance 2.x takes at most 9 images, 3 videos, and 3 audio files there, and audio needs at least one image or video reference on every model. New integrations should use /v1/videos. For still images, POST /v1/images also takes input_references; see Image generation API with reference images.

How are reference-to-video jobs billed?

Video jobs are reserved on submit at the provider's list price × 1.25 on every model, plus a 5.5% agent fee by default. Rates are the pricing_skus in GET /v1/videos/models; see also API pricing.

Two catalog notes: on minimax-h3, the first five reference images carry no extra charge and each additional image adds a per-image charge, while minimax-h3-max bills reference-to-video as output seconds.