Two-Proportion Z-Test Calculator
Compare two proportions or conversion rates with the two-proportion z-test. Enter successes and sample sizes for each group to get the z statistic, p-value, decision at your significance level, and a confidence interval for the difference.
Compare Two Proportions or Conversion Rates
Two-Proportion Z-Test
Where:
- \(x_1, x_2\) — number of successes (conversions, responders, defects, etc.) in each group
- \(n_1, n_2\) — sample sizes of the two groups
- \(\hat p_1, \hat p_2\) — sample proportions in each group
- \(\hat p\) — pooled proportion, the best estimate of the common proportion under the null hypothesis \(H_0: p_1 = p_2\)
- \(z\) — test statistic, compared against the standard normal distribution
The z statistic measures how far the observed difference \(\hat p_1 - \hat p_2\) is from zero, in units of its standard error computed under the assumption that the two population proportions are equal. Large absolute values of \(z\) indicate that the observed difference is unlikely to be due to sampling variation alone.
The confidence interval for the difference uses the unpooled standard error:
\[ (\hat p_1 - \hat p_2) \pm z_{\alpha/2} \sqrt{ \frac{\hat p_1 (1 - \hat p_1)}{n_1} + \frac{\hat p_2 (1 - \hat p_2)}{n_2} } \]
Two-Proportion Z-Test – Calculation Example
Example: an A/B test compares two landing pages. Version A (group 1) records 45 conversions out of 500 visitors; version B (group 2) records 25 conversions out of 500 visitors. We test \(H_0: p_1 = p_2\) against \(p_1 \neq p_2\) at \(\alpha = 0.05\).
\[ \hat p_1 = \frac{45}{500} = 0.09, \qquad \hat p_2 = \frac{25}{500} = 0.05, \qquad \hat p = \frac{45 + 25}{500 + 500} = 0.07 \]
\[ z = \frac{0.09 - 0.05}{\sqrt{0.07 \times 0.93 \times \left( \frac{1}{500} + \frac{1}{500} \right)}} = \frac{0.04}{0.0161} = 2.4788 \]
The two-sided p-value is \(2\left(1 - \Phi(2.4788)\right) = 0.0132\), which is below \(\alpha = 0.05\).
Result: \(z = 2.4788\), \(p = 0.0132\); reject \(H_0\). The conversion rates differ significantly (9.00% vs 5.00%, a +4.00 percentage-point difference). The 95% confidence interval for \(p_1 - p_2\) runs from +0.85 to +7.15 percentage points.
How to use the calculator
- Under Group 1, enter the number of Successes (e.g. 45) and the Sample size (e.g. 500). Successes cannot exceed the sample size.
- Under Group 2, enter the same information for the second group (e.g. 25 successes out of 500).
- Choose the Alternative hypothesis: two-sided (\(p_1 \neq p_2\)) for any difference, or one-sided if you predicted the direction in advance.
- Set the Significance level \(\alpha\) — 0.05 is the conventional choice; 0.01 or 0.10 are common alternatives.
- Read the results: sample proportions, their difference, the z statistic, the p-value, the decision at your \(\alpha\), and a confidence interval for \(p_1 - p_2\).
What is the two-proportion z-test?
The two-proportion z-test answers the question: is the difference between two proportions large enough to conclude the underlying population proportions differ? It is the standard tool when you observe a binary outcome in two independent groups — conversion rates in an A/B test, response rates in two treatment arms, defect rates from two production lines, or support for a policy among two demographics.
The logic is straightforward. Even if both populations had exactly the same proportion, the two sample proportions would rarely be identical because of random sampling variation. The test quantifies how much of a difference sampling variation alone would typically produce (the standard error) and compares the observed difference against that yardstick. The resulting z statistic follows approximately a standard normal distribution, which converts it into a p-value — the probability of seeing a difference at least this large if the null hypothesis of equal proportions were true.
Interpreting the output: if the p-value is at or below your chosen \(\alpha\), the difference is statistically significant and the null hypothesis is rejected. The confidence interval is just as informative: it gives the plausible range for the true difference \(p_1 - p_2\). If it excludes zero, the difference is significant; its width tells you how precisely the data pin down the effect. A significant but tiny difference (e.g. 0.2 percentage points) may matter commercially less than a non-significant but large one — significance and practical importance are different things.
For a quick check of how to read p-values in general, see our P-Value Calculator.
Assumptions, limitations and practical tips
- Independent samples. The two groups must consist of different, unrelated subjects. If the same subjects are measured twice (e.g. before/after), use McNemar's Test instead.
- Binary outcome. Each observation must be a success or failure — a count of conversions, cures, defects, and so on. Successes must be whole numbers not exceeding the sample size.
- Normal approximation. The z-test is an approximation that works when both groups have enough expected successes and failures under \(H_0\) (at least 5 each is a common rule of thumb, at least 10 for a stricter standard). With small samples or rare events, Fisher's Exact Test is more appropriate and does not rely on this approximation.
- Choose the alternative before looking at the data. A one-sided test is more powerful but only valid if the direction was genuinely predicted in advance. Switching to one-sided after seeing a result inflates the Type I error rate.
- Watch for zero standard error. If every observation has the same outcome (all successes or all failures), the pooled proportion is 0 or 1, the standard error collapses to zero, and the z-test is undefined — the calculator will flag this.
- Significance ≠ importance. With very large samples, trivial differences become significant. Judge the effect size through the confidence interval, and complement it with the Odds Ratio and Relative Risk Calculator for other ways to express the strength of the association.
- Related methods. The chi-square test of independence on the underlying 2×2 table gives an equivalent two-sided result; see the Chi-Square Test Calculator.
Frequently Asked Questions
How do I calculate a two-proportion z-test by hand?
First compute each sample proportion \( \hat p_1 = x_1/n_1 \) and \( \hat p_2 = x_2/n_2 \), then the pooled proportion \( \hat p = (x_1+x_2)/(n_1+n_2) \). The test statistic is \( z = (\hat p_1 - \hat p_2) / \sqrt{\hat p(1-\hat p)(1/n_1 + 1/n_2)} \). Look up \( z \) in a standard normal table: for a two-sided test, the p-value is \( 2(1 - \Phi(|z|)) \).
What is a good or normal z value for this test?
There is no "normal" z — it depends entirely on your data. But common benchmarks help: \( |z| \geq 1.96 \) gives a two-sided p-value of 0.05 or less, \( |z| \geq 2.576 \) gives p ≤ 0.01, and \( |z| \geq 1.645 \) gives p ≤ 0.10. A z near 0 simply means the two proportions are very close relative to sampling noise.
When should I use a one-sided vs. two-sided test?
Use a two-sided test (\( p_1 \neq p_2 \)) unless you committed to a specific direction before seeing the data. If you predicted in advance that group 1 would beat group 2, a one-sided test (\( p_1 > p_2 \)) is more powerful because it concentrates all of α in one tail. Choosing one-sided after peeking at the results artificially halves your p-value and inflates the risk of a false positive.
What's the difference between a z-test and a chi-square test on a 2×2 table?
For two-sided hypotheses with uncorrected chi-square, they are mathematically equivalent: \( z^2 \) equals the chi-square statistic, and the p-values are identical. The z-test is often preferred because the sign of z tells you which group has the higher proportion, and it extends naturally to one-sided tests and confidence intervals. Use Fisher's exact test instead when expected cell counts are below 5.
Can I use this test for percentages instead of raw counts?
The test requires counts: you need the number of successes and the sample size for each group. If you only know percentages, you must recover the counts — e.g. 9% of 500 visitors means \( 0.09 \times 500 = 45 \) conversions. If you don't know the sample sizes behind the percentages, the test is impossible, because the same percentage difference can be highly significant with large samples and pure noise with small ones.
Why does the calculator use different formulas for the test and the confidence interval?
The z statistic uses the pooled proportion because the null hypothesis \( p_1 = p_2 \) implies a single common proportion, giving the most accurate test under that assumption. The confidence interval uses the unpooled standard error \( \sqrt{\hat p_1(1-\hat p_1)/n_1 + \hat p_2(1-\hat p_2)/n_2} \), because the interval estimates the difference when the proportions are not assumed equal. This is standard practice and both are shown in the results.
What sample size do I need to detect a difference between two proportions?
Roughly \( n \approx (z_{\alpha/2} + z_{\beta})^2 \left[ p_1(1-p_1) + p_2(1-p_2) \right] / (p_1 - p_2)^2 \) per group. For example, detecting a change from 10% to 12% with 80% power at α = 0.05 requires about 3,800 observations per group. Small differences need surprisingly large samples — that's why tiny conversion-rate lifts in A/B tests often need tens of thousands of visitors.