Media tools

Video frames

Extract stills from one Sume-hosted clip at times you name. Durable image artifacts at source size. Charged in credits by its compute time.

Video frames uses one workspace media.sume.com clip and a program (at[] or fps) as input. It returns durable media.sume.com image artifacts. The source does not change. The server compiles ffmpeg on the worker media runtime (apps/api/src/routes.ts submitVideoFramesJob / getVideoFrames, apps/api/src/schemas.ts createVideoFramesSchema, and apps/worker/src/video-frames-executor.ts).

text

POST /v1/video-frames
GET  /v1/video-frames/:id

This family has no /v1/models/sume/…/runs alias. The resource ID is the job ID (request_id = video_frames_id).

A submit always returns 202. submitVideoFramesJob pins communicationMode: "async". Do not send mode: "sync". It does not give a 200. Poll this GET (or GET /v1/jobs/:id/status).

Hosted MCP: video_frames_create / video_frames_get (packages/mcp-server/src/mcp.ts). A write must have idempotency_key (and mcp:write under OAuth). Flow: video_frames_create → jobs_wait → video_frames_get.

This job does not use an admission seat (same class as a screenshot hop). At submit, it reserves its compute ceiling. Sume charges the job in credits for its own compute time, never more than the hold it reserved.

Create an extract

You must send video_url (a media.sume.com artifact or asset of this workspace) and only one of at[] or fps. The API does not fetch from the open internet. First, import the clip (POST /v1/media-imports). MCP writes must have Idempotency-Key. Also send it on REST, so that a retry does not queue a second extract.

Optional fields: format jpeg (default) or png, and max_edge 16–2160 (a long-edge clamp). If you do not send the clamp, the frames keep the source frame size.

bash

curl -X POST https://api.sume.com/v1/video-frames \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-frames-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "at": [0, 2.5]
  }'

If the submit is successful, the API returns 202 with request_id = video_frames_id (the job ID) and a video_frames object. Read the object later:

bash

curl https://api.sume.com/v1/video-frames/$REQUEST_ID \
  -H "Authorization: Bearer $SUME_API_KEY"

When resource_status is ready, frames[{t,url,width,height}] are durable artf_ images. If the extract fails for one instant, that frame has url null. This does not cause the job to fail. source_duration_seconds is the duration that the worker probed.

Program

Field	Effect
`at[]`	Explicit seconds. 1–24 values, each ≥ 0. Each value must be in the range `0 <= t < duration`. If not, the worker fails with `frame_time_out_of_range` and gives the probed duration.
`fps`	Sample rate, as an alternative to a list. `0 < fps ≤ 2`. Sume expands it to mid-bin samples (`0.5/fps`, `1.5/fps`, …), with a limit of 24 frames.
`format`	`jpeg` (default) or `png` (lossless inspection).
`max_edge`	Optional long-edge clamp, 16–2160. If you do not send it, the frames keep the source size (the restage path).

Send only one of at[] or fps. Limits: source ≤ 300 s (VIDEO_ANALYSIS_HARD_MAX_DURATION_SECONDS), and 24 frames for each call.

For evidence on the full clip (probe, eight mid-bin stills, optional STT), use video inspect. Inspect stills use a default max_edge of 768. This route does not use that clamp.

Refusals (stable codes)

Code	When
`ffmpeg_fields_rejected`	The client sent `vf` / `filter` / `filter_complex` / `select` / `ffmpeg` / `cmd` / `codec` / `crf` / `preset`. The server compiles the extract.
`400` (schema)	The request has `at[]` and `fps` together, or has none of them. Or it has more than 24 `at` values, `fps > 2`, or a `video_url` that is not on `media.sume.com`.
`frame_time_out_of_range`	An `at` value is outside `[0, duration)` (worker, after the probe).
`duration_out_of_range`	The source is longer than 300 s (worker).
`invalid_source_url`	The stored job has no `video_url` that the worker can use (worker).

The API rejects off-host URLs (https://example.com/…) at admit. Import the clip first.

When the source is longer than 90 s (probe reuse), warnings[] can include low_confidence_long_video. This warning does not cause the job to fail.

Not this surface

Need	Use
Probe / sampled stills / optional STT	Video inspect
A new MP4 cut	Video trim
Audio track as a durable wav / mp3	Audio detach
Pixel pass (dim / crop)	Video filter
Put several clips in a sequence	Timeline 1.0
Look at an unrendered HyperFrames composition	`hyperframes_check` then `hyperframes_snapshot`

Media tools

Video frames

Extract stills from one Sume-hosted clip at times you name. Durable image artifacts at source size. Charged in credits by its compute time.

text

POST /v1/video-frames
GET  /v1/video-frames/:id

This family has no /v1/models/sume/…/runs alias. The resource ID is the job ID (request_id = video_frames_id).

A submit always returns 202. submitVideoFramesJob pins communicationMode: "async". Do not send mode: "sync". It does not give a 200. Poll this GET (or GET /v1/jobs/:id/status).

Create an extract

Optional fields: format jpeg (default) or png, and max_edge 16–2160 (a long-edge clamp). If you do not send the clamp, the frames keep the source frame size.

bash

curl -X POST https://api.sume.com/v1/video-frames \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-frames-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "at": [0, 2.5]
  }'

If the submit is successful, the API returns 202 with request_id = video_frames_id (the job ID) and a video_frames object. Read the object later:

bash

curl https://api.sume.com/v1/video-frames/$REQUEST_ID \
  -H "Authorization: Bearer $SUME_API_KEY"

Program

Field	Effect
`at[]`	Explicit seconds. 1–24 values, each ≥ 0. Each value must be in the range `0 <= t < duration`. If not, the worker fails with `frame_time_out_of_range` and gives the probed duration.
`fps`	Sample rate, as an alternative to a list. `0 < fps ≤ 2`. Sume expands it to mid-bin samples (`0.5/fps`, `1.5/fps`, …), with a limit of 24 frames.
`format`	`jpeg` (default) or `png` (lossless inspection).
`max_edge`	Optional long-edge clamp, 16–2160. If you do not send it, the frames keep the source size (the restage path).

Send only one of at[] or fps. Limits: source ≤ 300 s (VIDEO_ANALYSIS_HARD_MAX_DURATION_SECONDS), and 24 frames for each call.

For evidence on the full clip (probe, eight mid-bin stills, optional STT), use video inspect. Inspect stills use a default max_edge of 768. This route does not use that clamp.

Refusals (stable codes)

Code	When
`ffmpeg_fields_rejected`	The client sent `vf` / `filter` / `filter_complex` / `select` / `ffmpeg` / `cmd` / `codec` / `crf` / `preset`. The server compiles the extract.
`400` (schema)	The request has `at[]` and `fps` together, or has none of them. Or it has more than 24 `at` values, `fps > 2`, or a `video_url` that is not on `media.sume.com`.
`frame_time_out_of_range`	An `at` value is outside `[0, duration)` (worker, after the probe).
`duration_out_of_range`	The source is longer than 300 s (worker).
`invalid_source_url`	The stored job has no `video_url` that the worker can use (worker).

The API rejects off-host URLs (https://example.com/…) at admit. Import the clip first.

When the source is longer than 90 s (probe reuse), warnings[] can include low_confidence_long_video. This warning does not cause the job to fail.

Not this surface

Need	Use
Probe / sampled stills / optional STT	Video inspect
A new MP4 cut	Video trim
Audio track as a durable wav / mp3	Audio detach
Pixel pass (dim / crop)	Video filter
Put several clips in a sequence	Timeline 1.0
Look at an unrendered HyperFrames composition	`hyperframes_check` then `hyperframes_snapshot`