The “mortality dropped from 34% to 26%” headline
A sepsis RCT abstract shouts: “28‑day mortality 26% vs 34% (p = 0.04).” Your instinct: “Great, 8% absolute reduction.” But what test gave that p? And is the 8% the only way to express the effect?
In ICU literature, binary outcomes — mortality, AKI, VAP, delirium, ECMO weaning success — dominate. The statistics for comparing two proportions are χ² (or z‑test), Fisher’s exact, and the effect measures: risk difference (RD), risk ratio (RR), and odds ratio (OR). Knowing which was used, and how to interpret the effect measure, keeps you from misreading the clinical impact.
The hypothesis tests
χ² test of independence (or two‑proportion z‑test)
For two independent groups, each with a binary outcome, the χ² test asks: Are the two proportions the same in the underlying populations?
- Valid when expected cell counts ≥ 5 in all four cells of the 2×2 table.
- Equivalent to the two‑sample z‑test for proportions.
- p‑value is asymptotic; with large samples it is fine.
Fisher’s exact test
When expected counts are small (any cell < 5), the χ² approximation breaks down. Fisher’s exact computes the exact probability of the observed (or more extreme) table given the fixed marginal totals.
- Use it when events are rare, or sample size is small (e.g., < 20 per arm).
- Modern software (R, Stata, SPSS) computes it instantly even for moderate n.
- If a paper uses χ² on a table with 1, 2, or 3 events in a cell, the p‑value is unreliable.
Appraisal checkpoint: Look at the 2×2 table (or reconstruct it). If any expected count < 5 and the authors used χ² without mentioning Fisher’s exact, flag it.
Effect measures — which one tells the clinical story?
Measure | Formula | When it’s intuitive | Caveat |
Risk Difference (RD) | p₁ − p₂ | Absolute benefit/harm per patient treated. NNT = 1/RD. | Depends on baseline risk; not portable across populations with different baseline. |
Risk Ratio (RR) | p₁ / p₂ | Relative reduction/increase; portable across similar populations. | Overstates benefit when outcome is common (>20%). |
Odds Ratio (OR) | (p₁/(1−p₁)) / (p₂/(1−p₂)) | Required for logistic regression, case‑control studies, and meta‑analysis. | OR > RR when outcome is common; clinicians often misinterpret OR as RR. |
The ICU rule of thumb
- If control event rate ≤ 10%: OR ≈ RR (both valid).
- If control event rate 10–30%: OR overestimates the relative effect; prefer RR or RD.
- If control event rate > 30%: OR is substantially larger than RR; report RD and RR, avoid OR for clinical communication.
Example: 28‑day mortality 26% vs 34%
Effect | Calculation | Value | Clinical translation |
RD | 0.26 − 0.34 | −0.08 | 8 fewer deaths per 100 treated; NNT = 13. |
RR | 0.26 / 0.34 | 0.76 | 24% relative reduction. |
OR | (0.26/0.74) / (0.34/0.66) | 0.69 | “Odds of death 31% lower” — sounds bigger than 24%. |
If the paper reports only OR = 0.69 and the control rate is 34%, a clinician who thinks “OR = RR” will believe the relative reduction is 31%, not 24%. Always convert OR → RR if the outcome is common.
The 2×2 table you should be able to rebuild
Outcome (+) | Outcome (−) | Total | |
Intervention | a | b | n₁ |
Control | c | d | n₂ |
Total | a+c | b+d | N |
From this you can compute:
- p₁ = a/n₁, p₂ = c/n₂
- RD = p₁ − p₂
- RR = p₁ / p₂
- OR = (a·d) / (b·c)
- χ² = N(ad − bc)² / [(a+b)(c+d)(a+c)(b+d)]
- Exact p (Fisher) — let the software do it.
Always check: Do the numbers in the table match the text? Abstract says “26% vs 34% (n = 200 each arm)” → a = 52, c = 68. If the text says “p = 0.04” but your recomputed χ² gives p = 0.06, there is a discrepancy — maybe they used a continuity correction, or Fisher’s exact, or the numbers were rounded.
Confidence intervals for proportions
The Wald interval (p̂ ± 1.96√[p̂(1−p̂)/n]) is not recommended for small n or extreme p. Prefer:
- Wilson score interval (accurate for all n, p).
- Clopper‑Pearson (exact) for very small samples.
- Newcombe (hybrid Wilson) for the difference of two proportions.
If a paper reports a CI for RD, RR, or OR, check the method in the statistical analysis section. “95% CI calculated using the Wald method” for a small trial is a red flag.
Your appraisal checklist for two proportions
# | Question | Why it matters |
1 | What test was used? (χ², Fisher’s exact, logistic regression) | Small expected counts → Fisher’s; χ² on sparse data inflates Type I error. |
2 | What effect measure is reported? (RD, RR, OR) | RD gives NNT; RR is portable; OR overstates effect when outcome is common. |
3 | Was the baseline risk stated? (Control event rate) | Needed to convert OR → RR and judge clinical relevance. |
4 | Are CIs reported for the effect measure? (And by what method?) | Wald CIs can be anti‑conservative; Wilson/Exact preferred. |
5 | Does the CI cross the null? (RD includes 0; RR/OR includes 1) | If yes, the “significant” p may be fragile. |
Go deeper
- StatPearls – “Risk Ratio and Odds Ratio” (NBK559967): free, clear formulas, when to use each, conversion between them. https://www.ncbi.nlm.nih.gov/books/NBK559967/
- OpenIntro Statistics – Chapter 6 “Inference for categorical data” (free PDF): χ², Fisher’s exact, Wilson CI, with R code. https://www.openintro.org/stat/textbook.php
- BMJ Statistics Notes – “The odds ratio” (Altman & Bland): why OR ≠ RR when outcome is common, with worked ICU example. https://www.bmj.com/content/bmj_stats_notes
- PMC2765277 – “Relative risk, odds ratio, and risk difference: which measure to use?” (free full text): thorough comparison with clinical scenarios. https://doi.org/10.1186/1471-2288-10-40
Next: Chapter 10 — Survival analysis I: censoring & Kaplan‑Meier