What the A/B Test Sample Size Calculator does
This calculator tells you how many visitors each arm of an A/B test needs to detect a given change in conversion rate, from the baseline rate, the minimum detectable effect, the significance level and the power you want. It handles one- and two-sided tests, several variants against one control (with Bonferroni or Sidak correction), unequal traffic splits, and converts the total into days from your traffic.
The formula is shown with your numbers substituted, so the answer can be checked by hand or against a statistics text.
How to use it
- Enter the baseline conversion rate of the control, from recent data for the same page and audience.
- Enter the smallest effect worth detecting, either in percentage points (10% to 12% is 2 pp) or as a relative change (a 20% lift on 10%).
- Choose alpha, power and whether the test is two-sided. Two-sided at 0.05 with 80% power is the most common starting point.
- Set the number of treatment variants, the correction for multiple comparisons, the allocation ratio and, optionally, daily eligible traffic and the share of it in the test.
- Read the per-arm sample, total and duration, and use the table below to see what a smaller or larger effect would cost.
Reading the results
The sample is per arm. With one control and two variants you need the control sample plus two variant samples.
Sample size scales with 1 / effect squared: halving the minimum detectable effect roughly quadruples the visitors needed. Pick the smallest effect that would change a decision, not the smallest you can imagine.
Duration is rounded up to whole weeks in the advice because behaviour differs by weekday. A test that reaches its sample on a Tuesday should still run to the end of the week.
Worked example: detecting 10% to 12% with 80% power
Baseline 10%, minimum detectable effect +2 pp (to 12%), alpha 0.05 two-sided, power 80%. Then z(0.975) = 1.959964 and z(0.8) = 0.841621. The pooled proportion is 0.11, so the null term is sqrt(2 x 0.11 x 0.89) = 0.442493 and the alternative term sqrt(0.10 x 0.90 + 0.12 x 0.88) = 0.442267.
n = (1.959964 x 0.442493 + 0.841621 x 0.442267)^2 / 0.02^2 = 1.536338 / 0.0004 = 3,840.85, rounded up to 3,841 per arm and 7,682 in total. The simpler unpooled formula gives 3,839. At 2,000 eligible visitors a day with all of them in the test, that is 4 days - so run a full week.
Formulas and scoring rules
- Pooled (default)
n1 = [z_a sqrt(pbar(1-pbar)(1 + 1/k)) + z_b sqrt(p1(1-p1) + p2(1-p2)/k)]^2 / (p2 - p1)^2pbar = (p1 + k p2) / (1 + k); Fleiss, Levin and Paik, without continuity correction.- Unpooled
n1 = (z_a + z_b)^2 (p1(1-p1) + p2(1-p2)/k) / (p2 - p1)^2- Critical values
z_a = z(1 - alpha/2) two-sided or z(1 - alpha) one-sided; z_b = z(power)Standard normal quantiles via Acklam's approximation refined to double precision.- Multiple variants
Bonferroni alpha' = alpha / m; Sidak alpha' = 1 - (1 - alpha)^(1/m)m = number of treatment variants compared with control.- Allocation and total
n2 = k x n1 per variant; total = n1 + m x n2- Duration
days = total / (daily visitors x share in test)Every sample size and day count is rounded UP to a whole number.
Limitations: what the result does not prove
- It assumes independent visitors and a fixed sample decided in advance. Sequential monitoring, bandits and Bayesian stopping rules need different calculations.
- The normal approximation is poor for very low baselines with small samples; if the expected conversions per arm are under about 10, treat the figure as a lower bound.
- It plans for one primary metric. Guardrail and secondary metrics each carry their own error rates.
- Traffic estimates are usually optimistic: bots, returning visitors and ineligible users all shrink the real sample.
Privacy: where your data goes
Everything you paste, type or drop is processed in this browser tab. It is not uploaded, logged, stored or sent to analytics. Session recording and tag-manager scripts are switched off on this page.
Standards and sources
- NIST/SEMATECH e-Handbook of Statistical Methods - checked 19 Sep 2026
- NIST/SEMATECH e-Handbook - Sample sizes required for proportions
- NIST/SEMATECH e-Handbook - Normal distribution critical values
Frequently asked questions
How many visitors do I need for an A/B test?
It depends on the baseline rate and the smallest effect you care about. For a 10% baseline and a 2 percentage point lift at 95% confidence and 80% power, it is 3,841 visitors per arm using the pooled formula.
What is minimum detectable effect?
The smallest true difference the test is designed to detect with the chosen power. Smaller effects can still show up, but with less than your planned probability; larger ones are detected more reliably.
Should I use a one-sided or two-sided test?
Two-sided unless you would genuinely take the same action whether the variant is equal or worse. One-sided tests need fewer visitors but cannot tell you when the change made things worse.
How does testing several variants change the sample size?
Each extra comparison adds a chance of a false positive, so alpha is split across them and each arm needs more visitors. Sidak is slightly less conservative than Bonferroni; both are shown in the formula line.
Does an uneven traffic split need more visitors?
Yes. For a fixed total, a 50/50 split gives the most power. Sending more traffic to one arm limits risk but raises the total sample needed, which the calculator shows when you change the allocation ratio.
Why does my result differ from another calculator?
Calculators use pooled or unpooled variance, with or without continuity correction, and sometimes simulation. Differences of a few percent are normal. This page states its formula and lets you switch between pooled and unpooled.
How long should an A/B test run?
Until it reaches the planned sample and covers at least one full weekly cycle, ideally two. Stopping early because results look good is the most common way tests produce false winners.
Last reviewed by the A2Z.Tools team against the sources listed above.
