Translate caption cues but keep the timings: a reading-speed check

Keep each cue's start and end, swap only the text, and flag lines that become too dense. A short Python check catches it before you pay for the render.

4 min readSume
All posts

When you translate burned-in captions, keep each cue's start and end from the source language and replace only text. A translation that is much longer than the original stays on screen for the same time, so measure characters per second before you submit the caption job.

Why the timing stays fixed

cues in POST /v1/video-captions are phrase-level overlay cards with text, start, end in seconds. They skip speech-to-text, so Sume does not retime them to the translated speech; the times you send are the times that burn.

That makes the translation step your responsibility: the same number of cues in the same order, one translated string each.

A check you can run

The function below refuses a mismatched cue count and prints any line above a characters-per-second limit. The 17 default is an arbitrary budget for the example, not a Sume rule; set your own.

def localize(cues, texts, max_cps=17):
    if len(cues) != len(texts):
        raise ValueError("cue count differs from translation count")
    out = []
    for cue, text in zip(cues, texts):
        cps = len(text) / (cue["end"] - cue["start"])
        if cps > max_cps:
            print(f"too dense ({cps:.1f} chars/s): {text!r}")
        out.append({**cue, "text": text})
    return out

en = [
    {"text": "Meet the new bottle.", "start": 0.0, "end": 2.0},
    {"text": "Keeps drinks cold for a day.", "start": 2.0, "end": 4.5},
]
es = ["Conoce la nueva botella.", "Mantiene las bebidas frías todo el día."]
print(localize(en, es))

When a line is too dense

Shorten the translation first. If the phrase cannot be shortened, widen its window by moving the neighbouring cue's boundary, and keep the cues non-overlapping. Re-run the check, then submit one caption job per language with Idempotency-Key set so a retry stays safe.

Sources

Related posts

More in Use cases

All Use cases posts

Written by Sume