Storyboard to finished video over Sume MCP: the tool order

Over hosted MCP, the order Sume's own server guidance gives for a multi-shot video is: generate and approve stills, generate each wordless clip from its approved still, probe the clips, then assemble with timeline_create, dry-running first. Paid steps are the stills, the clips and the render; probing and frame extraction are unbilled.

This post follows that order and notes where a client should stop and look, because the expensive mistakes in storyboard work happen early, before any clip is generated.

What is the order of tools?

Write calls need an idempotency_key, and under OAuth the mcp:write scope. If a client connected with Write off, the paid tools answer insufficient_scope instead of spending anything.

Why are approved stills the gate?

The server guidance says to inspect and accept each still before video spend. A still is the cheap artifact: if the product label is wrong or the character differs from shot to shot, you fix it with an image regeneration, not a clip regeneration. Once a still is approved, the same image goes into input_references for a wordless clip, which is how a character or product carries across shots.

On the REST side of the same models, frame_images with first_frame (and last_frame where the model supports it) is the image-to-video path. If both frame_images and input_references are sent, frame_images wins and the request is image-to-video. Check each model's supported_frame_images and supported_input_references in the model list before choosing.

What about shots where someone speaks?

The same guidance is blunt: video models do not lip-sync, so a speaking face is not a generate_video job with narration laid under it. A talking shot is made as speech first, then a talking-clip job on the accepted still. Only wordless beats go through generate_video. In a storyboard, mark each panel as wordless or spoken at the start so you route it correctly.

All of it is then assembled with one continuous audio spine and video[] shots, each with start, duration and optional transition. Every source_url and audio.url must be Sume-hosted, so mirror outside media first with media-imports_create.

What should the client check before the render?

Run the dry run and read the result. It shows duration_seconds, segment_count and estimated_cost_usd_micros. It cannot predict warnings about short sources, so probe real durations first with video_inspect, which is read-only and free by default (transcription bills the speech-to-text rate if you request it).

Then pull a few stills at the seams with video_frames_create: the last second of one shot and the first second of the next. at values must be inside the clip or the worker fails with frame_time_out_of_range. When the render finishes, warnings[] are soft: padded or looped short sources, snapped transitions and an ignored still motion do not fail the job and do not need a re-render.

Two last notes. Sume does not accept ffmpeg arguments anywhere in this chain; ffmpeg_fields_rejected is the refusal. And every paid write should carry its own idempotency_key, so a retried call returns the original job instead of billing a second time.