Testing Difference Between Two Proportions Using z-Test

Every time a news report says “men are three times more likely than women to exercise” or “students in private schools have higher smartphone access than those in government schools,” someone ran a statistical test behind that claim. When you want to know whether two groups genuinely differ on some yes/no characteristic – exercises or doesn’t, owns a smartphone or doesn’t, favours a policy or doesn’t – the two-proportion z-test is the tool that turns a gut feeling into a defensible conclusion.

What comparing two proportions actually means 🔗

A proportion is simply the share of a group that has a particular trait. If 81 out of 1,000 surveyed people exercise regularly, the sample proportion is 0.081, or 8.1%. When you have two independent groups – say, men and women, or students in urban and rural schools – you often want to know if their proportions are genuinely different, or if the gap you see in your sample is just random noise.

This is where the two-proportion z-test comes in. It uses sample data to test a claim about the difference between two population proportions, denoted p₁ and p₂.

Estimating proportions from sample data 🔗

You never get to measure an entire population directly, so you rely on samples. If x₁ people out of a sample of n₁ show the trait you’re studying, the sample proportion is p̂₁ = x₁/n₁. This value estimates the true population proportion p₁. The same logic applies to the second group: p̂₂ = x₂/n₂ estimates p₂.

A real-world example: the exercise gap 🔗

India’s Time Use Survey data offers a good illustration. According to an analysis of the survey, men were found to be nearly three times more likely to exercise than women, and participation was noticeably higher in cities than in villages. Among women specifically, urban participation stood at 8.1%, compared to just 3.1% in rural areas. These two numbers – 0.081 and 0.031 – are sample proportions. The natural question a researcher asks next is whether this gap reflects a real difference in the population or could simply be sampling variation.

The standard error of the difference between proportions 🔗

To test whether a gap is meaningful, you need to know how much random variation to expect just by chance. This is captured by the standard error of the difference between two proportions:

σ(p̂₁ − p̂₂) = √(p₁q₁/n₁ + p₂q₂/n₂), where q = 1 − p.

This formula combines the variability from both samples. Larger sample sizes shrink the standard error, which makes it easier to detect real differences – a pattern confirmed by standard treatments of the topic, including the two-proportion confidence interval framework taught in introductory statistics courses. In practice, since the true population proportions p₁ and p₂ are unknown, you substitute the sample proportions p̂₁ and p̂₂ when constructing confidence intervals, and a pooled proportion when running the hypothesis test itself – a distinction worth remembering, since mixing the two up is a common error.

Two conditions you must check first 🔗

A z-test is only valid when the underlying assumptions hold. Skipping this check is one of the most frequent mistakes students make, so it’s worth treating as a mandatory first step rather than a formality.

Independence of samples 🔗

The two groups being compared must be independent of each other. Men and women in a national survey, or students from private schools and students from government schools, qualify as independent groups because being in one group has no bearing on membership in the other. If the same individuals appeared in both groups, or the sampling design linked the two sets of observations, the standard z-test formula would no longer apply.

Minimum expected counts 🔗

The second requirement is that n₁p₁ and n₁q₁ must both be at least 5, and the same must hold for n₂p₂ and n₂q₂. This condition guarantees that the sampling distribution of the difference in proportions is close enough to a normal curve for the z-test’s probability calculations to be trustworthy. Consider a real example: an education survey found that smartphone availability at home was 79% among private school children compared to 63.7% among government school children. If each group’s sample included even a few hundred students, both n·p and n·q would comfortably clear the threshold of 5, so a z-test comparing these two proportions would be appropriate.

Building the z-test statistic step by step 🔗

The hypothesis test typically starts with a null hypothesis that the two population proportions are equal (p₁ = p₂) against an alternative that they differ. Because the null hypothesis assumes equality, the test statistic uses a pooled proportion p̂ that combines both samples’ successes and totals, rather than treating each sample’s proportion separately. This approach is standard practice in two-proportion hypothesis testing, since assuming equal proportions under the null hypothesis means there is really only one shared proportion to estimate.

Worked example: urban versus rural exercise rates 🔗

Let’s put real percentages to work. Suppose a researcher surveys 1,000 urban women and 1,000 rural women and finds exercise rates matching the national pattern: p̂₁ = 0.081 for urban women and p̂₂ = 0.031 for rural women. That gives x₁ = 81 and x₂ = 31.

First, check the conditions: n₁p̂₁ = 81, n₁q̂₁ = 919, n₂p̂₂ = 31, and n₂q̂₂ = 969 – all comfortably above 5. Independence also holds, since the two groups are separate individuals.

Next, calculate the pooled proportion: p̂ = (81 + 31) / (1000 + 1000) = 112/2000 = 0.056. Using this pooled value in the standard error formula gives a standard error of roughly 0.0103. Dividing the observed difference (0.081 − 0.031 = 0.050) by this standard error produces a z-statistic of approximately 4.86 – far beyond the typical critical value of 1.96 used for a 95% confidence level. This tells us the gap between urban and rural women’s exercise rates is highly unlikely to be due to chance alone.

Confidence intervals for the difference in proportions 🔗

A hypothesis test tells you whether a difference is statistically significant, but a confidence interval tells you the likely size of that difference. The formula is:

(p̂₁ − p̂₂) ± z_α/2·√(p̂₁q̂₁/n₁ + p̂₂q̂₂/n₂)

Notice that this formula uses the two sample proportions separately rather than a pooled value, since there’s no assumption here that the proportions are equal – you’re estimating how far apart they actually are.

Interpreting the interval 🔗

Continuing the worked example, a 95% confidence interval for the difference between urban and rural women’s exercise rates works out to roughly 0.030 to 0.070, or 3 to 7 percentage points. Since this range does not include zero, the difference is statistically significant – consistent with the z-test result above. If the interval had spanned both negative and positive values, including zero, you would conclude that the data does not provide convincing evidence of a real difference between the two groups.

Where this test shows up beyond the classroom 🔗

Two-proportion z-tests are everywhere once you start looking. Market researchers use them to compare brand preference across demographic segments. Public health officials use them to test whether an intervention changed vaccination or screening rates in one district compared to another. Education researchers rely on them to compare outcomes across school types or states. Even something as basic as computer ownership – which remains below 10% of households nationally, despite far higher mobile phone penetration – becomes a candidate for this kind of comparison when researchers ask whether ownership differs meaningfully between, say, students and non-students, or between different states.

Common mistakes to avoid 🔗

Using unpooled proportions in a hypothesis test: Remember that the hypothesis test’s standard error uses the pooled proportion, while the confidence interval uses each sample’s own proportion. Mixing these up gives incorrect standard errors and misleading conclusions.

Skipping the condition check: If n·p or n·q falls below 5 for either group, the normal approximation breaks down, and an alternative method – such as an exact test – becomes more appropriate.

Confusing statistical significance with practical importance: A very large sample size can make even a tiny, practically meaningless difference appear statistically significant. Always look at the actual size of the gap, not just whether the test crossed a significance threshold.

Ignoring the independence assumption: If the two groups overlap or influence each other’s measurements, the standard z-test formulas no longer hold, and a different statistical approach is needed.

Bringing it all together 🔗

The two-proportion z-test gives you a structured way to move from “these percentages look different” to “this difference is unlikely to be due to chance.” Whether you’re comparing exercise habits across genders, smartphone access across school types, or any other yes/no outcome between two independent groups, the same logic applies: estimate the proportions, check your conditions, compute the standard error, and let the z-statistic or confidence interval guide your conclusion.

What do you think? If you were designing a survey to compare two groups on a habit or behaviour relevant to your own life – say, screen time, savings habits, or dietary choices – what sample sizes would you need to feel confident about detecting a real difference? And how would you explain to someone unfamiliar with statistics why a small percentage-point gap can still be “significant”?

References

1. https://www.indiaspend.com/health/9-in-10-indians-do-not-exercise-977727
2. https://online.stat.psu.edu/stat200/book/export/html/193
3. https://news.careers360.com/aser-2021-sharp-spike-in-smartphone-availability-over-26-students-dont-have-access
4. https://stattrek.com/hypothesis-test/difference-in-proportions
5. https://www.dataforindia.com/computers/