A/B Test Sample Size Calculator
Sample size is the number of visitors each variant needs before your A/B test results can be trusted. Running a test without calculating sample size first is the most common mistake in CRO — it produces false positives that send teams shipping changes that don't actually improve conversion. Enter your baseline CVR and target improvement to find out how many visitors you need per variant.
Sample Size Calculator
Calculate how many visitors you need per variant for a statistically valid A/B test
How to Use This Calculator
- Baseline conversion rate — your current CVR on the page you're testing. Get this from your analytics platform for the last 30–90 days.
- Minimum detectable effect (MDE) — the smallest relative improvement you want to detect. 10–20% is typical. A smaller MDE means a larger required sample size. If you're testing a major redesign, 20%+ is reasonable. Testing a minor copy change, 10% is more appropriate.
- Statistical confidence — 95% is the industry standard (5% false positive risk). Use 99% only for irreversible changes. 90% is acceptable for low-stakes tests on low-traffic pages.
- Statistical power — 80% is standard (20% false negative risk). Use 90% when missing a true winner is especially costly.
Understanding the Results
The calculator returns the sample size per variant. Total visitors needed = sample size × number of variants. For a standard A/B test (control vs one variant), double the per-variant number.
The MDE comparison table shows how sample size changes at different MDE levels — useful for deciding what effect size is realistic given your traffic volume. Pair this result with the A/B Test Duration Calculator to find out how many days the test will take.
Frequently Asked Questions
How many visitors do I need for an A/B test?
It depends on baseline CVR, MDE, power, and confidence. At 3% baseline CVR, 15% MDE, 95% confidence, 80% power: ~8,000–10,000 visitors per variant. Use the calculator above — generic rules of thumb produce wrong results for your specific scenario.
What is minimum detectable effect (MDE)?
MDE is the smallest relative improvement you want to detect. 10% MDE on a 3% baseline means you're designing the test to detect a variant CVR of 3.3%+. Smaller MDE = larger required sample = longer test. Most practitioners start at 15–20% MDE as a practical baseline.
What happens if I test with too few visitors?
Your false positive rate explodes. At 95% significance with proper sample size, 5% of tests show a winner that isn't real. Stopping early when a test looks good can push that rate to 26%+. Under-powered tests waste time and ship changes that don't convert.
What is statistical significance in A/B testing?
95% significance means there's a 5% probability the observed difference happened by chance. It doesn't confirm the improvement is real or large — it only says it's probably not noise. Always reach both significance AND your pre-calculated sample size before declaring a winner.
Need Help Running Rigorous A/B Tests?
I've run 800+ A/B tests. A free CRO audit includes a full assessment of your testing setup, sample size methodology, and highest-leverage test candidates.
Get a Free CRO Audit