A/B Test Calculator
Runs a two-proportion z-test and a chi-square test on two conversion rates with Wilson confidence intervals, and computes the sample size for a target effect and power.
Two conversion rates in, and a verdict on whether the difference is real: a two-proportion z-test, a chi-square test and Wilson confidence intervals for each rate. It also computes the sample size you need before you start, which is the calculation that would have saved most of the tests people stop early.
What to do
Observed
A (control)
B (variant)
Test result
Significant (p = 0.0074 < 0.05)
A conversion rate
20.00%
95% interval 17.64% – 22.59% (Wilson)
B conversion rate
25.00%
95% interval 22.42% – 27.78% (Wilson)
Difference (B − A)
+5.00%pp
interval 1.35%pp – 8.65%pp
Relative lift
+25.0%
Two-proportion z-test (two-sided)
p̄ = 0.225 · SE = 0.01867 · z = 2.677 · p = 0.0074
Chi-square test (1 df)
χ² = 7.168 · p = 0.0074
Yates-corrected χ² = 6.885 · p = 0.0087
How to use it
Put in the two groups
Visitors and conversions for each. Raw counts, not rates — the counts are what carry the uncertainty.
Read the interval before the p-value
The Wilson interval says where each true rate plausibly lies. Two intervals that overlap heavily are two rates you cannot yet tell apart, whatever the p-value says.
Plan the next test with the sample size panel
Give it your baseline rate and the smallest lift worth detecting, and it tells you how many visitors per group you need. Running until it looks significant is not a test.
Common questions
What does p < 0.05 actually mean?
That if there were genuinely no difference, a gap this large or larger would turn up less than one time in twenty. It is not the probability that B is better, and it says nothing about how much better.
Can I stop as soon as it goes significant?
No. Peeking repeatedly and stopping at the first significant moment inflates the false-positive rate well past 5% — with daily checks over a fortnight it is closer to 20%. Fix the sample size first and run to it.
Why is my required sample size so large?
Because the effect you asked to detect is small. Halving the detectable lift roughly quadruples the sample needed. If the number is impossible, the honest conclusion is that this test cannot answer that question, not that the number is wrong.
Another calculator gives half this sample size
It is running a one-sided test, or using a different α, or omitting the continuity correction. This one is two-sided at α = 0.05 with the Fleiss, Levin and Paik correction, and it prints the formula so you can see which assumption differs.
Feed it from
Descriptive Statistics
Computes mean, median, mode, variance, standard deviation, quartiles and outliers from pasted numbers.
Unit Converter
Converts length, weight, area, volume, temperature, speed, pressure and data units, including Korean units.
Number Base Converter
Converts between binary, octal, decimal, hexadecimal and any base up to 36, with bitwise views.
Percentage Calculator
Works out percentage of, percentage change, discount, markup and reverse percentage in one place.
Next
These accept what this tool produces, so the result can go straight in.
Quick Chart
Draws a bar, line, area, pie or scatter chart from a table with round-number axes and a legend, and exports it as SVG or PNG.
Regression & Correlation
Fits a linear, polynomial, exponential, logarithmic or power model to paired data and reports R², Pearson r, Spearman ρ and residuals.
Descriptive Statistics
Computes mean, median, mode, variance, standard deviation, quartiles and outliers from pasted numbers.
Currency Converter
Converts between about thirty currencies at the ECB’s daily reference rates, crossed through the euro, or at a rate you type in.
Date Difference
Counts days, weeks, months and years between two dates, with a D-day view and business-day option.
Percentage Calculator
Works out percentage of, percentage change, discount, markup and reverse percentage in one place.