Video editing MCP server: let an AI agent trim, filter, and assemble

Sume's hosted MCP server at https://mcp.sume.com/mcp works as a video editing MCP server: an agent can inspect a clip (video_inspect), pull exact frames (video_frames_create), cut a range (video_trim), dim or crop it (video_filter), split off its audio (audio_detach), put a still and a video in one frame (timeline_compose), join or split audio (timeline_audio), and render one MP4 from an audio spine and ordered clips (timeline_create). Sume compiles ffmpeg on its own workers, so the agent never sends ffmpeg arguments.

The tool set comes from MCP tools and gates and each tool's docs page, read on 2026-09-26. The same tools are HTTP endpoints, and the basics page says the CLI and hosted MCP still work but are not part of the primary integration path today. Connecting a client is covered in Connect Claude Code, Cursor, or Codex to Sume.

Which MCP tools edit video?

Each edit runs as a job, and most follow the same path: submit, jobs_wait, then read the result.

What media can the agent edit?

Every editing tool takes media already on this workspace's media.sume.com, which is where Sume returns generated outputs, so the result of an earlier Sume job works as input. There is no open-internet fetch: off-host URLs such as https://example.com/… are rejected at admit, and hosted MCP cannot read files from your laptop.

The tool pages say to import outside media first. On MCP the import tool is media-imports_create, which the docs call the social URL mirror: a paid tool whose description currently limits it to public TikTok and Instagram video URLs, so it is not a general upload path.

How does an agent run an edit end to end?

Every write takes its own idempotency_key and, under OAuth, a session with mcp:write. The tools currently take the HTTP request body inside payload, next to idempotency_key, as in the video_trim call below. A cut-and-assemble pass looks like this:

• Inspect the source with video_inspect. frames: false returns probe facts only, enough to check probe.has_audio.
• Cut each range with video_trim: start plus exactly one of end or duration. The new MP4 goes into timeline_create video[] with source_in 0.
• Check a dim or crop with video_filter and check_only: true, then encode.
• Detach the voice once with audio_detach, then split it with timeline_audio if you need reusable pieces. A join needed only inside one render belongs on audio.parts[].
• Render with timeline_create. video[0].start must be 0, and transitions go only on slots after the first.
• For many same-shaped calls, such as one trim per clip, script_run runs them from one script.

{
  "idempotency_key": "trim-intro-001",
  "payload": {
    "video_url": "https://media.sume.com/artifacts/artf_demo/talk.mp4",
    "start": 2,
    "duration": 8
  }
}

What does editing cost over MCP?

Billing follows each tool's docs page:

• video_inspect probe and stills are unbilled; transcribe: true is billed per audio minute.
• video_frames_create is unbilled, and video_filter with check_only: true is free.
• video_trim, the video_filter encode, audio_detach, timeline_compose, and timeline_audio are billed per job. The docs say to confirm those rates live in GET /v1/catalog.
• A timeline render is listed at $0.10 per output minute on API pricing, and it reserves ceil(audio.duration_seconds / 60) minutes.
• Trim, filter, detach, compose, timeline audio, and the render run no provider inference: worker ffmpeg only. Paid calls accept dry_run for a cost preview and max_spend_usd as a cap.

What can't the agent do with these tools?

The docs draw these lines. Per-tool refusal codes for trim, filter, and detach are in Trim, filter, or detach audio from a video.

• Send ffmpeg arguments. On inspect, frames, trim, filter, and detach, fields such as filter, ffmpeg, cmd, or codec fail with ffmpeg_fields_rejected; the Timeline tools reject keys such as filtergraph, codec, or crf with a 400.
• Put trim, setpts, drawtext, subtitles, movie, lut3d, or anything that reads a file or socket in a filtergraph.
• Find a caption create in tools_list. The caption create tools are unlisted on hosted MCP, though still callable by name for in-flight clients; only video-captions_get is listed. Over HTTP, captions are POST /v1/video-captions: see how to burn captions onto a video.
• Dim or crop inside a render. A pixel pass is not a render option, so run video_filter first.
• Exceed the caps: trim sources up to 1800 seconds and cuts up to 900; filter and frames sources up to 300; compose output up to 300; renders of 1–1800 seconds.