Media tools

Video filter

Dim, crop, or apply an allowlisted pixel filtergraph to one Sume-hosted clip. Returns a new MP4. Material preparation, not timeline placement.

Video filter 1.0 uses one workspace media.sume.com clip and a validated program (ops[] dim/crop and/or a filters-only filtergraph) as input. It returns a new MP4. The source does not change. The server compiles ffmpeg on the worker media runtime (apps/api/src/routes.ts createVideoFilterV1 / submitSumeVideoFilterJob, and packages/timeline-compiler/src/filter.ts compileVideoFilterProgram).

text

POST /v1/video-filter
POST /v1/video-filter/check              # unbilled contract check
POST /v1/models/sume/video-filter-1.0/runs   # same body, no extra `model` field

There is no GET /v1/video-filter/:id. Poll the job envelope:

text

GET /v1/jobs/:id/status
GET /v1/jobs/:id/result

Hosted MCP: video_filter (packages/mcp-server/src/mcp.ts). A write must have idempotency_key (and mcp:write under OAuth). Flow: video_filter with check_only: true (free) → video_filter → jobs_wait → jobs_result.

Check a program (unbilled)

POST /v1/video-filter/check runs the same checks as the encode: the schema, the op whitelist, the filtergraph allowlist, and the Sume-host / HEAD source preflight. It returns diagnostics and not a 400. It does not create a job, reserve credits, boot a box, or use the encoder. A program that passes this check can still fail on the box (bad expression, memory, time). In that case, you get a structured job error.

Idempotency-Key is not necessary for the check. A valid response is object: video_filter_check. It contains valid, encode: "not_run", diagnostics[], and the compiled program.filters (names only, no argv). If the program is valid, it also contains an estimate. It also contains next_action submit_video_filter | fix_program_and_recheck.

bash

curl -X POST https://api.sume.com/v1/video-filter/check \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'

Encode

You must send video_url (a media.sume.com artifact or asset of this workspace) and a program. The program must contain a minimum of one of these: ops[] or a non-empty filtergraph. The API does not fetch from the open internet. First, import the clip (POST /v1/media-imports). You must send Idempotency-Key.

Optional fields: ops, filtergraph, metadata, and the usual mode / webhook_url / wait_timeout_seconds communication fields.

The default mode is async (readCommunicationOptions). To wait a maximum of 30 seconds for a 200 with the completed job, send mode: "sync". If the job does not complete in that time, you get 202, and then you poll the job.

bash

curl -X POST https://api.sume.com/v1/video-filter \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-filter-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'

If the submit is successful, the API returns a job (request_id is the job ID). When the status is result_ready, GET /v1/jobs/:id/result returns kind: video_filter. The result contains video_url (a new artf_, never the source), duration_seconds, ops_applied, filtergraph, the compiled filters[], and optional warnings[]. Put that MP4 into timeline_create video[].

Each encode job uses a flat number of credits. The API rate card lists the current credits. The check is free. There is no provider inference. Only the worker ffmpeg runs.

Program

Field	Effect
`ops[]`	Ordered program. Sume applies it before `filtergraph`. Maximum 8. You must send a minimum of one op or a filtergraph.
`ops[].op: "dim"`	Multiplies the luma of the full clip. `amount` is in (0, 1]. `0.45` makes the clip less bright, and `1` makes no change. The API refuses `0` and `>1`. Black stays black, and the chroma does not change.
`ops[].op: "crop"`	Rectangle as fractions of the source frame. `x` and `y` are in `[0, 1]`. `width` and `height` are in `[0.05, 1]`. `x+width ≤ 1` and `y+height ≤ 1`. The compiler rounds to even values for yuv420p.
`filtergraph`	Filters-only ffmpeg graph. Sume applies it after `ops[]`. Maximum 2048 characters and 32 named filters. No inputs, no outputs, and no paths. The server adds `[0:v]…[vout]` around the graph. Internal labels (`split[a][b]`) are permitted. Stream specifiers (`[0:v]`) are not permitted.

Limits (from packages/api-contract/src/index.ts): source ≤ 300 s (compose-band clip ceiling). The output keeps the geometry, frame rate, and audio of the source, unless the program changes them.

The allowlisted filter names are in packages/timeline-compiler/src/filter.ts VIDEO_FILTER_GRAPH_ALLOWED_FILTERS (tone, blur, geometry, fade, internal compositing). These items are not on that list: trim / setpts (use video trim), drawtext / subtitles / movie / lut3d, and all filters that read a file or a socket.

Refusals (stable codes)

Code	When
`video_filter_ops_empty`	The request has no `ops[]`, and `filtergraph` is missing or empty.
`video_filter_too_many_ops`	More than 8 ops.
`unsupported_filter_op`	`ops[].op` is not `dim` or `crop`. Put other pixel work in `filtergraph`.
`unsupported_filter_op_field`	An op has an extra key (dim accepts only `{op, amount}`, and crop accepts only `{op, x, y, width, height}`).
`video_filter_amount_out_of_range`	The dim `amount` is not in `(0, 1]`.
`video_filter_crop_out_of_bounds`	The crop fractions are outside the source frame, or a side is less than `0.05`.
`invalid_filtergraph`	The graph is empty or too long, or it has an unknown filter, an illegal alphabet, or a stream specifier. `unknown_filter` gives the token and the allowlist.
`ffmpeg_fields_rejected`	The client sent `vf` / `filter` / `ffmpeg` / `cmd` / `codec` / `crf` / `-i` / related fields. The server compiles ffmpeg.
`unsupported_media_source`	`video_url` is not on the Sume media host.
`source_not_found`	The `media.sume.com` URL is dead or is not from this workspace.
`unsupported_media_type`	The HEAD result is not a video.
`source_too_large`	The source is larger than the timeline download budget.
`output_duration_exceeded`	The source is longer than 300 s (worker).

The API rejects off-host URLs (https://example.com/…) at admit. Import the clip first.

Not this surface

Need	Use
Probe / stills / optional STT	Video inspect
A new MP4 cut	Video trim
Audio track as a durable wav / mp3	Audio detach
Put several clips in a sequence	Timeline 1.0
Caption plates	HyperFrames compose / caption assembler
The accurate frame at `t`, at the source size	Video frames

Media tools

Video filter

Dim, crop, or apply an allowlisted pixel filtergraph to one Sume-hosted clip. Returns a new MP4. Material preparation, not timeline placement.

text

POST /v1/video-filter
POST /v1/video-filter/check              # unbilled contract check
POST /v1/models/sume/video-filter-1.0/runs   # same body, no extra `model` field

There is no GET /v1/video-filter/:id. Poll the job envelope:

text

GET /v1/jobs/:id/status
GET /v1/jobs/:id/result

Check a program (unbilled)

bash

curl -X POST https://api.sume.com/v1/video-filter/check \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'

Encode

Optional fields: ops, filtergraph, metadata, and the usual mode / webhook_url / wait_timeout_seconds communication fields.

bash

curl -X POST https://api.sume.com/v1/video-filter \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: video-filter-001" \
  -d '{
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "ops": [{ "op": "dim", "amount": 0.45 }]
  }'

Each encode job uses a flat number of credits. The API rate card lists the current credits. The check is free. There is no provider inference. Only the worker ffmpeg runs.

Program

Field	Effect
`ops[]`	Ordered program. Sume applies it before `filtergraph`. Maximum 8. You must send a minimum of one op or a filtergraph.
`ops[].op: "dim"`	Multiplies the luma of the full clip. `amount` is in (0, 1]. `0.45` makes the clip less bright, and `1` makes no change. The API refuses `0` and `>1`. Black stays black, and the chroma does not change.
`ops[].op: "crop"`	Rectangle as fractions of the source frame. `x` and `y` are in `[0, 1]`. `width` and `height` are in `[0.05, 1]`. `x+width ≤ 1` and `y+height ≤ 1`. The compiler rounds to even values for yuv420p.
`filtergraph`	Filters-only ffmpeg graph. Sume applies it after `ops[]`. Maximum 2048 characters and 32 named filters. No inputs, no outputs, and no paths. The server adds `[0:v]…[vout]` around the graph. Internal labels (`split[a][b]`) are permitted. Stream specifiers (`[0:v]`) are not permitted.

Refusals (stable codes)

Code	When
`video_filter_ops_empty`	The request has no `ops[]`, and `filtergraph` is missing or empty.
`video_filter_too_many_ops`	More than 8 ops.
`unsupported_filter_op`	`ops[].op` is not `dim` or `crop`. Put other pixel work in `filtergraph`.
`unsupported_filter_op_field`	An op has an extra key (dim accepts only `{op, amount}`, and crop accepts only `{op, x, y, width, height}`).
`video_filter_amount_out_of_range`	The dim `amount` is not in `(0, 1]`.
`video_filter_crop_out_of_bounds`	The crop fractions are outside the source frame, or a side is less than `0.05`.
`invalid_filtergraph`	The graph is empty or too long, or it has an unknown filter, an illegal alphabet, or a stream specifier. `unknown_filter` gives the token and the allowlist.
`ffmpeg_fields_rejected`	The client sent `vf` / `filter` / `ffmpeg` / `cmd` / `codec` / `crf` / `-i` / related fields. The server compiles ffmpeg.
`unsupported_media_source`	`video_url` is not on the Sume media host.
`source_not_found`	The `media.sume.com` URL is dead or is not from this workspace.
`unsupported_media_type`	The HEAD result is not a video.
`source_too_large`	The source is larger than the timeline download budget.
`output_duration_exceeded`	The source is longer than 300 s (worker).

The API rejects off-host URLs (https://example.com/…) at admit. Import the clip first.

Not this surface

Need	Use
Probe / stills / optional STT	Video inspect
A new MP4 cut	Video trim
Audio track as a durable wav / mp3	Audio detach
Put several clips in a sequence	Timeline 1.0
Caption plates	HyperFrames compose / caption assembler
The accurate frame at `t`, at the source size	Video frames