AI dubbing workflow on Sume: detach, transcribe, TTS, timeline

Sume does not ship a single dubbing call. You chain four of its steps around a translation step you supply: detach the audio, transcribe it, translate the text, synthesize the new language with TTS, and render the original picture over the new voice with Timeline. For a 60-second video that is about $0.16 on Sume, plus whatever your translator costs.

The chain

Keeping the length close

Translated text rarely matches the original runtime. Measure the new voice's duration after synthesis and set the timeline's audio.duration_seconds to match; the picture slots follow the audio spine. If the new voice runs long, trim words from the translation and rerun TTS: at under a nickel for a minute, a retake is cheap.

curl -X POST https://api.sume.com/v1/timeline-1.0/render \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: dub-es-001" \
  -d '{
    "audio": { "url": "https://media.sume.com/artifacts/artf_demo/voice-es.wav", "duration_seconds": 58 },
    "video": [
      { "source_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4", "start": 0, "duration": 58 }
    ]
  }'

What this does not do

It does not match lips. The docs are explicit that video models do not lip-sync to generated TTS or to a later voiceover, so dubbing suits voiceover-style footage, screen recordings and B-roll, not close-ups of a person speaking. Add captions afterwards in the target language for another $0.20.

Source rules

Audio detach reads a media.sume.com source only (source up to 1800 s) and costs $0.01 per job. Timeline also takes Sume-hosted URLs, so import the original video first and use the resulting media.sume.com URL in both steps.