How to change the background of a video with AI

To change the background of a video with AI, give a video-to-video (edit) model the clip and a prompt that names the new setting and says what must stay the same, such as “Replace the background with a beach at sunset. Keep the person and their movements the same.” The model returns a rewritten clip. You send no mask and need no green screen, but nothing guarantees the person comes back identical, so check the result.

On Sume, that edit is the video-to-video mode of gemini-omni-flash-1.1. The facts below come from the Video Router docs, the Sume API reference, and Sume's API code, read on 2026-09-28.

How do I change a video's background with Sume?

Send the clip's public HTTPS URL as video_url to POST /v1/video-router/generate with model: "gemini-omni-flash-1.1", currently the only model with a video-to-video edit, and put the new background in prompt. Edit a video with a prompt covers the other fields, why the edit uses this route, and how to fetch the result.

curl -X POST https://api.sume.com/v1/video-router/generate \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: background-edit-001" \
  -d '{
    "model": "gemini-omni-flash-1.1",
    "prompt": "Replace the background with a beach at sunset. Keep the person and their movements the same.",
    "video_url": "https://example.com/talking-shot.mp4",
    "resolution": "720p",
    "mode": "async"
  }'

What should the prompt say?

Write it as an instruction about the clip you already have: first the change, then what to keep. Sume's docs use that shape in their own edit example, “Replace the bottle with an apple. Keep everything else the same.” For a background, name the place, the time of day, and the light, then list what must not change: the person, their clothes, their movements, and the camera. Some examples to adapt:

• A talking shot: “Replace the office behind her with a quiet park in the morning. Keep her, her clothes, and her gestures the same.”
• A sky replacement: “Replace the gray sky with a clear sunset sky. Keep the buildings, the street, and the people the same.”
• A product clip: “Put the bottle on a marble kitchen counter instead of the plain table. Keep the bottle, its label, and the camera move the same.”

Will the person and the rest of the clip stay the same?

Not guaranteed. Sume's API reference describes the edit as a source clip “rewritten by prompt”, and the docs don't promise that the person, their face, or their movements come back unchanged. Watch the edited clip next to the original before you use it.

Some things aren't documented at all: a maximum length for the source clip, and whether the output keeps the source's length, framing, or original sound. Two things are documented: the model always generates audio, with no switch to turn it off, and aspect_ratio is rejected in edit mode, so the output's shape isn't something you set.

Should I edit the video or start from a photo?

Edit the video when the motion must be your own footage. If a still is enough, or a new generated shot is fine, start from a photo instead. For the last route, host the edited still at a public HTTPS URL first: the Image API returns signed links, and video requests refuse signed URLs.

How much does it cost?

An edit is billed per second of output, by resolution, at the provider's list price × 1.25: $0.125 a second at the default 720p, plus a 5.5% agent fee by default. The amount is reserved from your workspace balance when you submit. How much does an AI video cost? lists the model's rates at its other resolutions.