Translate video subtitles by API: English to Korean captions

To translate an English video's subtitles into Korean with the Sume API, get timed English sentences from speech-to-text, translate each sentence, and send the Korean lines as timed cues to POST /v1/video-captions with a Hangul style such as black-outline. Cues skip speech-to-text, so each Korean line is burned exactly at the times you give.

Facts come from the Video inspect, Video captions, and Agent Completions docs and the STT 1.0 and caption schemas in the Sume API reference, read on 2026-09-27. Captioning Korean speech is a different job, covered in Korean subtitles API.

Which calls make up the pipeline?

Three steps. The first and the last are fixed Sume calls; the middle one can be any translator.

How do I get timed English sentences?

Send language_code: "en" as the speech-to-text hint and segmentation: { "mode": "sentence" }. Both transcription calls take that pair and return gapless sentence segments[] shaped like caption lines. Auto-generate subtitles shows the full inspect request and transcript fields.

• Use video inspect when the English video is already a media.sume.com clip in your workspace, such as the output of an earlier Sume job. A clip with no audio track fails with inspect_source_has_no_audio.
• Use STT 1.0 when you have the audio at a public HTTPS audio_url; its schema prefers a Sume media URL. duration_seconds (1–600) sizes the reservation; omit it to reserve one minute.

How do I translate the lines without losing the timing?

Send only the sentence texts to your translator and keep each segment's start and end in your own code, in the same order. Pair every Korean line with its segment, and check that the counts match before you burn.

To keep the whole pipeline on Sume, ask the Sume agent through Agent Completions. Put the English lines in input, which the agent treats as data, never as instructions; bind output_schema to a lines array; and set generation_spend_cap_usd, which has no default. The call answers 202 with a run receipt, and you poll the run until output holds the lines.

• The key needs agent_completions:write. Keys created before Agent Completions shipped lack it and get 403 insufficient_scope.
• The cap must be above 0, and it is enforced against the run's generation spend. The agent's own LLM turn is billed outside it, so the cap is not the run's total cost.

curl -sS -X POST https://api.sume.com/v1/agent/completions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: subtitles-ko-001" \
  -d '{
    "instruction": "Translate each English line in the input into Korean subtitles. Keep the order and the count.",
    "input": { "lines": ["Today we are introducing the new dashboard.", "Setup takes one minute."] },
    "output_schema": {
      "name": "acme/subtitles-ko/v1",
      "schema": {
        "type": "object",
        "additionalProperties": false,
        "required": ["lines"],
        "properties": { "lines": { "type": "array", "items": { "type": "string" } } }
      }
    },
    "generation_spend_cap_usd": 1
  }'

How do I burn the Korean lines onto the video?

Send the video's public HTTPS URL as video_url and one cue per line: text, plus start and end in seconds. Sume burns exactly that copy at those times without transcribing the English audio again. A long Korean line can take a newline in text for a two-line card. cues cannot be combined with words, segments, or script_text.

Name a Hangul style such as black-outline, as Korean subtitles API explains: the Latin styles refuse Korean copy, and in the current code an omitted style is chosen from the letters of all your cues together, so translations heavy with English names can fall back to the Latin slam.

curl -X POST https://api.sume.com/v1/video-captions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: captions-ko-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "style": "black-outline",
    "cues": [
      { "text": "오늘은 새 대시보드를 소개합니다.", "start": 0, "end": 3.2 },
      { "text": "설정은 1분이면 끝납니다.", "start": 3.2, "end": 6.0 }
    ]
  }'

What are the limits, and what does it cost?

The caption step sets most of the limits:

• Cue start and end run from 0 to 60 seconds, and the current caption worker refuses a source video longer than 60 seconds with duration_out_of_range. Cut a longer video first, as in Split a long video into short clips.
• One job takes 1–200 cues, and each cue's text is 1–400 characters.
• SRT uploads are unsupported; send the lines as cues.
• video_url must be a fetchable public HTTPS video. Localhost, private-network, non-HTTPS, and signed or private URLs are rejected.
• Transcripts bill at the STT 1.0 rate, $0.01 per audio minute on API pricing, plus a 5.5% agent fee by default; an inspect's probe is unbilled. Each caption job reserves the fixed amount on the Video captions page for videos up to 60 seconds, plus the same fee.