A/B test two reference sets on Seedance 2.5: six takes for about $16

To compare two reference sets on Seedance 2.5, keep the prompt, duration and resolution identical, run each set three times, and score the six clips blind. At 10 seconds and 480p each take costs about $2.69 on Sume, so the experiment is about $16.12. One take per set proves nothing, and the Video Router docs list no seed parameter, so you cannot repeat a generation exactly.

Why three takes each

Generative video varies from run to run. If set A beats set B on a single take, you may only have seen luck. Three takes per set gives you a rough sense of the spread without a large bill. It is still a small sample, not a statistical test, so treat a clear win as a hint and a close result as a tie.

Design the experiment

Change only the reference images. Everything else stays fixed. Pass reference images as reference_image_urls. The Video Router docs say limits differ per model, so read capabilities from GET /v1/video-router/models for the reference cap before you build a set.

• Set A: three tight portraits of the subject.
• Set B: one portrait, one full-body shot, one environment shot.
• Prompt: identical text with no mention of which set it is.
• Settings: seedance-2.5, 480p, 10 seconds, same aspect ratio.
• Scoring: a colleague who does not know the sets rates each clip 1 to 5 on likeness, motion and artifacts.

Submit the six jobs safely

Give every job its own Idempotency-Key, such as ab-a-1 through ab-b-3, so a retry after a network error does not create a duplicate bill. If your plan has a low processing concurrency, extra jobs wait as queued, which the generation admission docs describe as normal, not a failure. Free allows one processing job and five queued.

for s in a b; do
  for take in 1 2 3; do
    curl -s -X POST https://api.sume.com/v1/video-router/generate \
      -H "Authorization: Bearer $SUME_API_KEY" \
      -H "Content-Type: application/json" \
      -H "Idempotency-Key: ab-$s-$take" \
      -d "{\"model\":\"seedance-2.5\",\"prompt\":\"The chef from the references plates a dish.\",\"reference_image_urls\":[\"https://example.com/$s-1.png\",\"https://example.com/$s-2.png\",\"https://example.com/$s-3.png\"],\"resolution\":\"480p\",\"duration\":10,\"mode\":\"async\"}"
  done
done

Read the result

Download the six MP4s, shuffle the file names, and score. Then run the winner once at 720p to confirm it holds up, which is a single $5.78 clip. If the sets tie, choose the cheaper one to prepare. Poll each job as described in Jobs and results rather than resubmitting.