AI clip to a vertical Short, in order: captions go last

What order turns an AI clip into a finished vertical Short?

Generate the clip in 9:16, probe it, trim it, dim it only if captions will sit on bright footage, add a music bed in a Timeline 1.0 render, and burn the captions on the finished file as the last step. Captions are pixels once burned, so every step that follows them can damage or crop them, and every step before them is free to change the picture without touching the text.

The order below uses only Sume surfaces documented today. YouTube's side is simple: its upload page says a Short can be up to 3 minutes with a square or vertical aspect ratio, and that you can upload up to 15 files at once in Studio on a computer.

The seven calls

Rates are the documented public rates at the time of reading. Steps with no fixed rate in the docs are marked, and are left out of the total.

Why each step sits where it does

Probe right after generation. The probe reports duration, width, height, rotation and whether the clip has audio. That last field decides step 7: if the clip is silent, speech-to-text captions fail with caption_no_speech, and the answer is to pass authored cues. A music bed does not add speech, so a Short that is only footage and music also needs cues.

Trim before anything else is paid for. Video trim with the default exact precision is frame accurate; keyframe can start a GOP early. Dim after trimming so you are not filtering frames you cut. The video filter dim multiplies luma by an amount in (0, 1], and 0.45 darkens it noticeably. Use POST /v1/video-filter/check before the encode.

Assemble before captioning. Timeline 1.0 defaults to 1080 by 1920 and fits every slot with cover, which crops. Captions burned earlier can be cropped or scaled by that fit. The music is also easier to place in the render: audio.mode: "silence" declares the length with no spine, and a soundtrack then supplies the bed. The docs say duck_db needs a real spine, so a music-only Short cannot be ducked.

The caption call, and what restyling costs

The captions request takes the render's video_url and, for a silent or music-only file, authored overlay cues with text, start and end in seconds. A later restyle with source_caption_id reuses the source video and word timings, but it is still a render, so the docs treat it as another job.

Add the documented fixed-rate steps for a Short with all of them: $0.02 + $0.02 + $0.125 + $0.10 + $0.20 = $0.465. Skip the optional dim and the total is $0.445. Both exclude generation, the probe and any stills you extract with video frames, which are billed by their own compute.

curl -X POST https://api.sume.com/v1/video-captions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: short-final-captions" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/short.mp4",
    "style": "slam",
    "cues": [
      { "text": "Made in one take", "start": 0.4, "end": 2.6 },
      { "text": "Now with sound", "start": 2.8, "end": 5.0 }
    ]
  }'

What can go wrong

The Short ends up over 60 seconds: the fixed caption price is stated only up to 60 seconds, so check the catalog before you batch longer clips. The captions are hard to read: dim and re-run, then restyle. The file is the wrong shape: probe it, because YouTube's page only treats square or vertical videos as Shorts.

For the pieces in more depth, read the posts on dimming before captions and on silent clips and overlay cues.