Gemini editing in YouTube Shorts: how does it work?
YouTube says you prompt a first draft, then refine by reordering, trimming pauses and syncing music. Here is which of those steps map to separate API calls.
YouTube describes it as conversational editing: you start with prompts for a first draft, then refine it by reordering, trimming pauses and syncing music to the beat. YouTube says Gemini Omni was integrated into Shorts and the YouTube Create app earlier in 2026. The announcement does not give a technical breakdown of what runs behind each request.
The YouTube facts are from its Made on YouTube post, read 2026-09-29. The API side is Sume's video trim, Timeline, Music 1.0 and video captions docs.
What does YouTube say the editing flow is?
Three named refinements after the first draft. YouTube also says that in August 2026 alone, hundreds of thousands of channels used the feature per day.
| Refinement YouTube names | Nearest Sume call |
|---|---|
| Reordering | Order the video[] slots in POST /v1/timeline-1.0/render |
| Trimming pauses | POST /v1/video-trim for one range; a transcript pass to find pauses |
| Syncing music to the beat | No single call; see the beat-sync post |
| First draft from prompts | POST /v1/videos for a clip |
How is an API different from a chat editor?
In the app, one conversation carries the whole edit. Through an API each step is its own job: trim returns a new MP4 and leaves the source untouched, and Timeline takes an audio spine plus an ordered video[] list and returns one MP4. Nothing remembers your last instruction, so your script or agent holds the plan.
Which steps can I script today?
Trimming and joining are both documented. Video trim takes a start and one of end or duration and costs $0.02 per job. Timeline 1.0 is $0.10 per output minute, rounded up. For pause removal from a talking clip, remove silence from video walks through it.
What about captions and music?
Captions are burned with POST /v1/video-captions, with styles such as slam and punch. Music comes from Music 1.0, which rejects a duration field, so you fit length in the edit rather than the request. YouTube's post does not describe either as part of the Gemini flow.
How do I get a 9:16 first draft by API?
Sume's video docs list gemini-omni-flash-1.1 at 3–10 seconds in 16:9 or 9:16 with native synced audio. Whether that is the model behind YouTube's feature is not something YouTube's post says. For availability of the app feature itself, see Gemini Omni in YouTube Create.
Can an agent run the whole edit for me?
Sume's hosted MCP lists video_trim and timeline_create as tools, so an agent can call the same steps one by one: trim, order the slots, render. Trim returns a job you wait on with jobs_wait and read with jobs_result. That gives you a chat-shaped loop, though the plan and the taste calls stay with whoever drives it.
Sources
Related posts
More in Use cases
- GPT Image 2.5 comments on an image: review notes as mask edits
Sume's image API has no comment field. To act on several notes about one picture, send one masked edit per note and feed each result into the next request.
- Sketch to image with GPT Image 2.5 API: send a drawing as a reference
To turn a sketch into a finished image with GPT Image 2.5 on Sume, send the drawing in input_references and describe the result. The request, size tips, limits.
- Gym promo video with AI: from your own gym photos
Make a gym promo video without a shoot: animate photos of your own floor and classes, add a voiced offer, music and captions, and render a vertical cut.
- Hair salon promo video with AI, from your own photos
Make a hair salon promo video from your own photos: animate the space and real finished looks, add a voiced offer and music, and render it vertical.
Written by Sume