Reference ingest uncertain[]: five kinds and what to do

The uncertain[] array in a Sume reference-ingest manifest is the only list of reasons to look at a frame again. Each entry has a kind, an optional ref and at time, and a suggested string. The contract defines five kinds: text_low_confidence, gradual_transition_candidate, ocr_language_unsupported, ocr_unavailable and audio_ambiguous.

Reference ingest is dest first (docs say development is auto-on and production is opt-in), so check tools_list for reference_ingest before you build on it. It reads one clip of up to 300 seconds that your workspace already holds on media.sume.com. That fits series episodes, which YouTube says are rolling out across web, mobile and TV (YouTube Blog, read 2026-10-04).

The five kinds

The reference ingest docs define the list as the only reason to re-inspect and do not describe each kind at length. The right column below is an inference from the kind names plus the documented rule; the manifest's own suggested string is the authority for each entry.

Look once per entry

The documented flow is: read the attached crop, then call video_frames_create at a manifest time, at most once per entry. video_frames takes 1 to 24 times per call, so one request can cover every entry that carries an at. Lines under the confidence threshold (default 0.85, set by ocr.min_confidence_attach_crop) come back as needs_verification with a crop, which is how most text_low_confidence entries are settled without a second call.

Collect the times

The sample manifest below is shortened to the fields this step reads. Replace it with the manifest your read returned.

uncertain = [
    {"kind": "text_low_confidence", "ref": "t3", "at": 2.4,
     "suggested": "read the crop"},
    {"kind": "gradual_transition_candidate", "at": 6.1,
     "suggested": "look at a frame near 6.1 s"},
    {"kind": "audio_ambiguous", "suggested": "listen to the track"},
]

times = sorted({
    round(item["at"], 3)
    for item in uncertain
    if "at" in item and item["kind"] != "audio_ambiguous"
})
others = [i["kind"] for i in uncertain if "at" not in i]

print("video_frames at:", times[:24])
print("no frame to pull:", others)

Frames come from video frames, which is billed by its Modal compute and reserves its ceiling at submit. If the clip is over 300 seconds, ingest answers source_too_long_for_reference_ingest; use video_inspect for that clip.