Add B-roll to a talking-head avatar video with the Sume API

To add B-roll to a talking-head avatar video with the Sume API, detach the avatar clip's audio and use it as the spine of a Timeline 1.0 render, then alternate video[] slots between the avatar clip and B-roll. The voice plays through every cutaway, and each return to the avatar sets source_in equal to the slot's start, so the lips land back on the right words.

The facts below come from the avatar video, Audio detach, Timeline 1.0, and Timeline compose docs and the Sume API reference, read on 2026-09-26. Anything described as current behavior is read from Sume's code. For the render document in general, see How to assemble a long-form video.

What do I need, and which calls are involved?

Every URL in the render must already be your workspace's media.sume.com artifact or asset. Here each one is the output of an earlier Sume job.

• A-roll: a talking avatar video from POST /v1/avatar-1.0/talking-video. One job covers an estimated 4–60 seconds, and completed results can include media.sume.com video artifacts.
• B-roll: Sume-hosted clips or stills from earlier Sume jobs. Sume's model docs say wordless beats, B-roll, and product motion come from image generation, inspect, then video generation, and that video models do not lip-sync to a later voice-over, so speaking shots stay on the avatar.
• Optional pauses: a multi-scene avatar video can include a voice.type: "silence" beat, a non-speaking beat with a required duration, which leaves a gap in the voice for a product shot.

How do I turn the avatar's voice into the spine?

Send the avatar clip's URL to POST /v1/audio-detach without a range. The default output is sample-exact wav at the source's rate and channels, the format audio.url expects, and the result's duration_seconds becomes audio.duration_seconds.

Skip the sample_rate: 16000 plus channels: "mono" speech-to-text shape here. The API reference says the output inherits the spine's sample rate and channel count, and a spine under 32 kHz warns audio_spine_low_fidelity.

How do I lay out A-roll and B-roll slots?

start is where a slot sits on the spine, and source_in is its in-point into the file. Because the spine is the avatar's own audio, spine time and the avatar clip's time are the same moment, so at a hard cut every avatar slot uses a source_in equal to its start. In the current compiler a slot's own audio is never mixed in, so B-roll sound never competes with the voice.

Keep those returns as hard cuts. With a transition, the current compiler starts the incoming clip the transition's length before its start, so a matching source_in would be start minus that length.

B-roll slots play from their own source_in (default 0). Per the Sume API reference, a clip shorter than its slot is filled by render.pad_mode; a still is a static hold.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: timeline-broll-001" \
  -d '{
    "audio": { "url": "https://media.sume.com/artifacts/artf_demo/avatar-voice.wav", "duration_seconds": 20 },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/avatar.mp4", "start": 0, "duration": 5 },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/broll-1.mp4", "start": 5, "duration": 4 },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/avatar.mp4", "start": 9, "duration": 6, "source_in": 9 },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/product.png", "start": 15, "duration": 2 },
      { "source_url": "https://media.sume.com/artifacts/artf_demo/avatar.mp4", "start": 17, "duration": 3, "source_in": 17 }
    ]
  }'

Can the presenter stay on screen during a cutaway?

Yes. Timeline compose puts one still, such as a product image, and one video in the same frame; its stack defaults give a half-banner with the still on top.

For a cutaway, set compose's video.source_in and video.duration to the stretch of the avatar clip you are covering, then put the result in video[] at that same start with source_in: 0: the composed clip already begins at the cut point. Match compose's output.width and output.height to your timeline.

Which frame rate and size will the edit use?

With output.fps omitted, the longest video sources decide the rate. In a mostly-avatar edit, B-roll at another rate is the material that gets resampled: a frame repeated or dropped every few frames, which shows as judder on motion and is reported as output_fps_resamples_sources.

Timeline's default output is 1080×1920, while avatar video resolution is currently 720p with a default aspect_ratio of 9:16. Set output.width and output.height to your delivery size, and give B-roll of another shape a fit: cover (the default), contain, stretch, or blur.

What does it cost?

The render's public rate is $0.10 per output minute on API pricing, and its reserve is ceil(audio.duration_seconds / 60) minutes, so the 20-second edit above reserves one minute. The avatar video, detach, and compose steps are separate jobs, each billed on its own.