Free A/B testing tool
Free Chi-Square Calculator
Enter visitors and conversions for 2 to 4 variants to run a chi-square test on your A/B test. You get the chi-square statistic, degrees of freedom, p-value and the conversion rate of each variant. Already have a chi-square value? Switch to the second tab to turn it into a p-value.
Live calculator
Run a chi-square test
Variant A
10.00%
Variant B
13.00%
P-value
0.0355
Chi-square = 4.422 with 1 degree of freedom
Significant at 0.05
- Chi-square statistic
- 4.422
- Degrees of freedom
- 1
Formulas
Expected = row total x column total / grand total
Chi-square = sum of (observed - expected)² / expected
df = (rows - 1) x (columns - 1)
No Yates continuity correction is applied.
What is the chi-square test?
The chi-square test of independence checks whether two things are related. In an A/B test the question is: does the variant a visitor saw change whether they convert? The test compares the counts you observed with the counts you would expect if every variant converted at the same rate. The bigger the gap, the bigger the chi-square statistic and the smaller the p-value.
How to calculate chi-square
Put your results in a table with one row per variant and two columns: converted and did not convert. Then:
- Find the expected count for each cell: row total x column total / grand total.
- For each cell, work out (observed - expected)² / expected.
- Add up all the cells. That sum is the chi-square statistic.
- Degrees of freedom = (rows - 1) x (columns - 1), which is variants - 1 for an A/B test.
- Look up the p-value in the chi-square distribution with those degrees of freedom.
Worked example: variant A had 1,000 visitors and 100 conversions, variant B had 1,000 visitors and 130 conversions. In total, 230 of 2,000 visitors converted and 1,770 did not. Each variant had 1,000 visitors, so the expected counts are 1,000 x 230 / 2,000 = 115 conversions and 885 non-conversions per variant.
- A converted: (100 - 115)² / 115 = 1.957
- A did not convert: (900 - 885)² / 885 = 0.254
- B converted: (130 - 115)² / 115 = 1.957
- B did not convert: (870 - 885)² / 885 = 0.254
The sum is a chi-square of 4.42 with 1 degree of freedom, which gives a p-value of about 0.035. That is below 0.05, so B's higher conversion rate (13% vs 10%) is significant. This calculator does not apply the Yates continuity correction.
Chi-square test vs z test
With two variants, the chi-square test and the two-proportion z test are the same test. The chi-square statistic equals z squared: in the example above z = 2.10, and 2.10² = 4.42. The p-values match too. Our A/B test significance calculator uses the z test and also shows the lift between variants. The chi-square test is most useful when you have three or four variants and want to know if any of them differ, in one test. If it is significant, compare the variants in pairs to find which one is different.
For other kinds of test statistics, use the p-value calculator.
Assumptions
- Each visitor is counted once, in one variant only.
- Visitors are independent: one visitor's choice does not affect another's.
- Every expected count is at least 5. With fewer, the chi-square p-value is not reliable. The calculator shows a warning when this happens. Collect more data or use Fisher's exact test instead.
- You decided the sample size before you looked at the results, not as soon as the p-value dipped below 0.05.
Plan how many visitors you need before you start with the sample size calculator.
FAQ
What does the chi-square p-value mean?
It is the chance of seeing differences between the conversion rates at least this large if all variants really convert the same. A p-value below 0.05 is the usual line for calling the result significant.
Should I use a chi-square test or a z test for an A/B test?
For two variants they give the same answer: the chi-square statistic is the square of the two-proportion z score, and the p-values match. Use the chi-square test when you have three or four variants and want one test across all of them.
Does this calculator use the Yates correction?
No. It runs the standard Pearson chi-square test without a continuity correction, which matches most A/B testing tools. The Yates correction makes the test more cautious and mostly matters for small samples.
How many degrees of freedom does a chi-square test have?
For a table of rows and columns, df = (rows - 1) x (columns - 1). An A/B test table has one row per variant and two columns (converted, did not convert), so df is the number of variants minus 1.
Run A/B tests that call the winner for you
PageDuel splits your traffic, tracks conversions and revenue, and tells you when a variant is a real winner.
14-day free trial · No credit card required