Video analyses

This page documents the legacy video_analysis / vana_ resource (typed scenes[]). This resource is not trend discovery or virality prediction, and it does not generate video. A reusable Format from clip structure is a remix job, not this analysis.

When create is still enabled, a video-understanding model (TwelveLabs Pegasus) watches the MP4 and segments it into typed scenes. Then ffmpeg extracts at least one JPEG still per second of each scene (keyframes), plus a representative keyframe_url. The current analysis_version is 1.1.

POST /v1/video-analyses          # dest: 410; prod until PR-C2
GET  /v1/video-analyses/:id      # stored rows, both environments
GET  /v1/video-analyses          # stored rows, both environments

The remote MCP wrappers video-analyses_create / _get / _list obey the same flag. Production lists them until PR-C2, and dest does not list them. Poll the remaining jobs with jobs_wait (usual runtime 3–5 minutes, poll every 30–60s).

Accuracy caveat: longer videos give less accurate results. Short clips give the most reliable results.

Create an analysis job (production until PR-C2)

Dest callers: do not send a POST to this endpoint. Use video inspect, or dest-only video_analyze / video_segment when Sume lists them.

The required field is video_url. The optional fields are max_scenes (2–40, default 24), include_transcript (when true, each scene's audio carries the spoken speech for that scene, and if not, audio is null), plus the usual mode / webhook_url / wait_timeout_seconds communication fields.

curl -X POST https://api.sume.com/v1/video-analyses \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-analysis-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/example/ad.mp4",
    "max_scenes": 24
  }'

A successful submit returns 202 with a vana_… resource id, a video_analysis job (request_id), and status_url / result_url.

If possible, use a durable media.sume.com URL from media imports (POST /v1/media-imports) or a completed Sume generation artifact. This surface has no media_id intake gate.

Credits note

Each accepted analysis reserves and captures a flat number of credits under the current fixed estimate. The API rate card lists the current credits.

Duration limits

Unsupported inputs

Poll and read

curl https://api.sume.com/v1/jobs/job_123/status \
  -H "Authorization: Bearer $SUME_API_KEY"

curl https://api.sume.com/v1/video-analyses/vana_123 \
  -H "Authorization: Bearer $SUME_API_KEY"

curl "https://api.sume.com/v1/video-analyses?limit=20" \
  -H "Authorization: Bearer $SUME_API_KEY"

When the resource is ready, it includes whole-video metadata plus a typed scenes[] array. Each scene has contiguous start_seconds / end_seconds, duration_seconds, summary, visual, optional shot_type / camera_motion, on_screen_text, and confidence (0–1). Each scene also has audio ({ speech, has_speech, has_music } when the request asked for include_transcript, otherwise null).

Each scene also has keyframe_url, the representative JPEG still. This still is the 1s sample nearest 40% of the scene, or the historical 40% pick on sub-1s scenes. The value can also be null with a keyframe_mirror_failed:scene_N warning. The keyframes field is an array of { t, url } stills for every whole second of that scene. A failed second is url: null plus keyframe_mirror_failed:scene_N:t_T, and it does not drop the scene.

The 300s hard duration cap is also the stills ceiling (~300 JPEGs). This surface has no separate 1-second text/context track.

Scopes

Related

• Video inspect (current clip inspection)
• Video captions
• Trending videos (only discovery metadata, not analysis)
• Jobs and results
• Media inputs