YouTube A/B tests skip Shorts: make two caption-style variants

Can you A/B test a Short on YouTube?

No. YouTube's help page on A/B testing titles and thumbnails says you cannot A/B test Shorts, scheduled lives or Premieres, and that once a video is transitioned to a Short you lose access to any tests created for it earlier. The same page describes a Short as a video under three minutes in a square or taller aspect ratio, so a re-cut that lands on either side of that line changes which tools you get.

If you want to know whether one caption look holds attention better than another on a Short, the built-in tool will not run the comparison for you. What you can do is produce the two versions cheaply and compare them yourself, knowing it is not a controlled test.

What the help page says

Only the statements below come from YouTube. Everything after this table is about Sume.

Re-burn the same Short in a second style

Sume's video captions route takes a finished clip and burns captions on it. To restyle a video you already captioned, pass source_caption_id instead of video_url. Sume reuses that caption's source video and the word timings it already has, so no second speech-to-text pass runs. Billing is unchanged: the docs say a restyle is still a render.

The docs give a fixed estimate of $0.20 per accepted standalone caption job for videos up to 60 seconds, and tell you to confirm it in GET /v1/catalog. On that basis, two caption looks of one 45-second Short cost 2 x $0.20 = $0.40 in caption jobs. That figure leaves out whatever you spent generating or trimming the clip itself.

# first look: transcribes and burns
curl -X POST https://api.sume.com/v1/video-captions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: short-look-a" \
  -d '{"video_url": "https://media.sume.com/artifacts/artf_demo/short.mp4", "style": "slam"}'

# second look: reuses the first job's video and word timings
curl -X POST https://api.sume.com/v1/video-captions \
  -H "Authorization: Bearer $SUME_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: short-look-b" \
  -d '{"source_caption_id": "vc_123", "style": "punch"}'

Change one thing between the variants

A comparison only means something if the two Shorts differ in one respect. The Latin-script styles named in the docs are slam, punch and tiktok-green. The design field can override one colour, size or placement value on a style, but the docs say it is not supported on punch or tiktok-green, so a single-variable test on a colour is easiest with slam plus a changed design.colors.active.

Keep the wording identical. If you pass words alongside source_caption_id you change the copy, which is a second variable.

Why this is not a real A/B test

Two uploads are two separate videos. They reach different viewers at different times, so a gap in views can come from the posting hour as easily as from the captions. Treat the numbers in YouTube Studio as a note for your next batch, not a verdict.

Also read the platform's guidance on near-duplicate uploads before you publish pairs of almost identical clips; the post on AI video variants and originality covers what the help pages say. For long-form videos, where the real tool does apply, see three cuts and thumbnails.