"How many creatives should I test?" gets answered with numbers — three, five, ten a week — and all of them are wrong, because the right answer depends on your budget, your win rate and how fast your winners fatigue.
It is not a number. It is a rate, and you can calculate yours.
Why a fixed number cannot be right
Testing a creative costs money: you must spend enough for the result to mean something. Call that the test cost.
Most creatives lose. Call the fraction that win your win rate.
So finding one winner costs roughly test cost ÷ win rate. With a 1-in-8 win rate and a $50 test, each winner costs $400 to find.
Winners do not last forever. Call how long one survives its useful life.
Your required testing rate is whatever keeps a winner in rotation as the old one dies. If a winner lasts three weeks and one in eight wins, you need to test roughly eight creatives every three weeks — not because eight is a magic number, but because that is what your own numbers demand.
Change any of the three inputs and the answer changes. That is why advice phrased as a number is useless.
How much spend a test needs
The most common testing mistake is calling results on too little data.
A rough floor: enough spend to produce at least 1,000 impressions and ideally 50+ link clicks before judging anything below the hook. At typical CPMs that is a small amount of money, but it is not $5.
What you can read earlier, and what you cannot:
| Metric | Readable after | Why | |---|---|---| | Hook rate (3s view) | ~1,000 impressions | High frequency event, stabilises fast | | Hold rate (completion) | ~1,000 impressions | Same | | Click-through rate | ~2,000 impressions | Less frequent | | Cost per purchase | Much more | Purchases are rare; this needs real spend |
This table is the practical core of creative testing. Hook rate stabilises long before conversion does, which means you can kill bad creative cheaply and only spend real money on the ones that survive the cheap filter.
A two-stage structure
Because of the table above:
Stage 1 — the cheap filter
Run every new creative on a small budget. Judge on hook rate alone. Kill the bottom half.
This costs very little per creative and removes most of them. A creative nobody watches past three seconds has not been tested for conversion — it has simply not been seen.
Stage 2 — the real test
Survivors get meaningful spend and are judged on cost per purchase.
Most of your budget lives here, spread across far fewer creatives. This is the stage where statistical patience actually matters.
The error most people make is running one stage: either judging everything on conversion (expensive, slow) or judging everything on hook rate (you ship pretty videos that do not sell).
What counts as a different creative
A real test needs a real difference. Changing the music is not a variant; it is a rounding error.
Meaningful variables, roughly in order of impact:
- The hook — the first three seconds. By a distance the biggest lever.
- The angle — which problem the ad claims to solve.
- The format — talking head versus faceless versus demonstration.
- The presenter — who is on screen, or whether anyone is.
- The setting.
- Everything else — pacing, music, captions, colour grade.
If you are testing ten things a week and all ten differ only at level 6, you are not testing. You are generating noise and paying for it.
A useful discipline: change one level at a time. Five hooks on the same body tells you about hooks. Five completely different videos tells you which of five videos won, which does not generalise to the sixth.
Calculating your own rate
Work this out once with your real numbers:
- Win rate. Over your last twenty creatives, how many reached acceptable cost per purchase? If it is 2, your win rate is 10%.
- Test cost. What you spend getting one creative to a readable result. Stage 1 plus, for survivors, stage 2.
- Useful life. How many weeks a winner runs before cost per purchase degrades past tolerance.
- Concurrent winners needed. How many live winners your spend level requires.
Then:
creatives to test per week
= (concurrent winners needed ÷ useful life in weeks) ÷ win rate
Needing two live winners, each lasting four weeks, at a 12% win rate: (2 ÷ 4) ÷ 0.12 ≈ 4 creatives per week.
The output is usually lower than the advice you have read, and it is yours rather than someone else's.
Raising the win rate beats raising the volume
Testing more is the brute-force answer. Winning more often is the better one, and it is mostly about where variation goes.
- Vary the hook hardest. Most of the spread in outcomes lives in the first three seconds.
- Mine your reviews. The phrasing customers actually use outperforms copywriting. Your review section is a list of pre-validated hooks.
- Keep losers' data. Patterns across failures are as informative as the winners, and they are free.
- Do not kill a winner because you are bored of it. Fatigue shows up in the numbers, not in your feelings about the video.
What makes volume affordable
The arithmetic above has an uncomfortable implication: if each creative costs a lot to produce, a low win rate makes testing prohibitive. At $300 per video and a 10% win rate, each winner costs $3,000 in production alone before media.
This is the actual argument for generated creative, and it is narrower than the marketing for it suggests. It does not make creative better. It makes the cost of an additional variant nearly flat, which turns a 10% win rate from a budget problem into a scheduling one.
Once a creative proves out and is carrying real spend, the economics flip and commissioning original production becomes sensible again. Faysell is built for the testing half specifically — several variants from one product photo, with the hook as the thing you vary.
The summary
There is no correct number of creatives. There is a rate, and it falls out of your win rate, your test cost and how long winners survive — all three of which you can measure in an afternoon.
Filter cheaply on hook rate, spend seriously only on survivors, and vary the first three seconds harder than anything else.