Replace or remove the audio in a video with the Sume API

To replace the audio in a video with the Sume API, render the clip as the only video[] slot of a Timeline 1.0 render (POST /v1/timeline-1.0/render) and pass the new track as audio.url: the output's sound comes from that spine, not from the clip. To remove the audio, cut the clip with POST /v1/video-trim and audio: "drop".

The facts below come from the Video trim, Audio detach, and Timeline 1.0 docs and the field descriptions in the Sume API reference, read on 2026-09-26. Anything described as current behavior is read from Sume's code. Detach's own fields are covered in Trim, filter, or detach audio.

Which call do I need?

Each tool takes a clip that is already your workspace's media.sume.com artifact or asset, such as the output of an earlier Sume job, and returns a new artifact; the source is untouched. Every create needs an Idempotency-Key.

How do I remove the audio from a video?

Trim the whole clip with audio: "drop": start: 0 and end at the clip's length. An end past the source clamps and warns trim_clamped_to_source, but one trim keeps at most 900 s: a longer range is refused with video_trim_range_empty. Trim's other options are covered in Trim, filter, or detach audio.

If you want a silent track rather than none, render the clip with audio.mode: "silence" instead.

curl -X POST https://api.sume.com/v1/video-trim \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-mute-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/clip.mp4",
    "start": 0,
    "end": 42,
    "audio": "drop"
  }'

How do I replace the audio with a new track?

Set audio.url to the new Sume-hosted track, such as a TTS master from an earlier Sume job, and audio.duration_seconds (1–1800) to the output length. Put the clip in video[0] with start: 0 and a duration that covers the spine. In the current compiler the clip's own audio is never mixed in.

• Keep the frame. Timeline's default output is 1080×1920 and the default fit is cover, which in the current code scales a clip of another shape to fill the frame and crops the rest. Set output.width and output.height (even integers, 256–2160) to the clip's size.
• New audio longer than the clip: per the Sume API reference, render.pad_mode fills the slot. auto (the default) loops short sources instead of holding a long freeze, loop replays the source, and freeze holds the last frame. The docs call padded or looped short sources soft warnings, not failures.
• Clip longer than the audio: shorten the slot. The docs let coverage trail the spine by at most 0.5 s, and in the current code a slot that ends later is refused with invalid_segment_timing.
• Use a spine-grade file. The output inherits the spine's sample rate and channels, and a spine under 32 kHz, such as video inspect's 16 kHz speech-to-text audio, warns audio_spine_low_fidelity.
• New speech does not move lips: Sume's model docs say video models do not lip-sync to a later voice-over. For new lines on a talking face, make a lip-synced clip.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: timeline-replace-audio-001" \
  -d '{
    "audio": {
      "url": "https://media.sume.com/artifacts/artf_demo/new-voice.wav",
      "duration_seconds": 30
    },
    "output": { "width": 1920, "height": 1080 },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/clip.mp4", "start": 0, "duration": 30 }
    ],
    "render": { "pad_mode": "freeze" }
  }'

How do I make the existing audio louder or quieter?

Detach the clip's audio with POST /v1/audio-detach, whose default wav is the format audio.url expects; a clip with no audio track fails with detach_source_has_no_audio. Then render the clip over that file with audio.gain_db, from −60 to 12 dB, applied to the spine; it defaults to 0 and is illegal with silence. To lay music under the voice as well, see how to add background music.

What does it cost, and why was my request refused?

The render's public rate is $0.10 per output minute on API pricing, and its reserve is ceil(audio.duration_seconds / 60) minutes. For trim and detach, the docs say to confirm the rate live in GET /v1/catalog.

• audio_url_required / audio_url_and_parts_exclusive: no spine and no silence, or both url and parts.
• silent_audio_takes_no_url / _parts / _gain / _source_in: a spine field sent with silence.
• unsupported_media_source / source_not_found: an off-host or dead URL.