The ICU paper on your tablet
It is 08:00. You open an ICU journal on your tablet — a new RCT: early prone positioning in ARDS. The abstract reports: “Mortality 28% vs 34%, p = 0.049, HR = 0.76 (95% CI 0.55–1.05). Kaplan‑Meier curves shown. Intention‑to‑treat analysis. No adjustment for confounders. No discussion of interaction with severity.”
Your task: evaluate whether this paper should change your practice. You have the tools from Chapters 1–15. Let’s apply them systematically.
The appraisal framework
Use the checklist below for any ICU paper:
1. Study type and design
- What question does the study answer? (Descriptive, analytical, interventional?)
- Is the design appropriate? (RCT for causality, cohort for association)
- Was randomisation and blinding reported? (Chapter 3)
2. Population and external validity
- Who was included? (Inclusion/exclusion criteria — does it match your ICU?)
- Where was the study done? (Single centre vs multicentre; community vs academic)
- Was the setting described in detail? (Chapter 2, Chapter 15)
3. Intervention and comparison
- What was the intervention? (Dose, timing, duration, delivery method)
- Was the comparison standard care or placebo? (Chapter 3)
- Was the protocol described fully? (Chapter 1, Chapter 3)
4. Outcomes and measurement
- What was the primary outcome? (Mortality, ICU stay, biomarker, functional status?)
- Was the outcome clinically meaningful? (Chapter 5)
- How was it measured? (Chart review, electronic health record, independent adjudication?)
- Was the measurement blinded? (Chapter 3)
5. Statistical analysis
- What analysis was primary? (ITT vs PP) — (Chapter 4)
- Was the analysis appropriate for the data? (Chapter 5, Chapter 6, Chapter 9)
- Were confounders adjusted? (Chapter 13)
- Was the sample size adequate? (Chapter 7)
- Were interaction and sensitivity analyses reported? (Chapter 13)
6. Results
- Are the effect measures reported with CIs? (Chapter 7, Chapter 9)
- Is the CI clinically meaningful? (Chapter 7)
- Are the results consistent with the design? (Chapter 2, Chapter 3)
7. Limitations and generalisability
- What limitations did the authors acknowledge? (Selection bias, confounding, missing data)
- Were sensitivity analyses performed? (Chapter 13)
- Does the study apply to your ICU? (Chapter 1, Chapter 15)
8. Decision
- Based on all the above, should you change practice?
- What further evidence is needed?
Applying the framework to the example
Using the abstract only (for demonstration):
- Design: RCT, but no details on randomisation or blinding (red flag).
- Population: Not described — we don’t know if the study applies to our ICU.
- Outcome: Mortality — clinically meaningful.
- Analysis: ITT reported (good), but no confounder adjustment, no interaction analysis.
- Results: HR 0.76 (CI 0.55–1.05) crosses 1 — not significant; p = 0.049 is fragile.
- Limitations: Not discussed — potential for selection bias, confounding, overfitting.
Conclusion: The abstract suggests a potential benefit (HR < 1), but the CI crosses 1, the analysis is minimal, and the study details are insufficient. Before changing practice, read the full paper, check the methods, and consider additional evidence.
Go deeper
- StatPearls – “Critical appraisal” (NBK540915): free, structured framework for evaluating clinical papers. https://www.ncbi.nlm.nih.gov/books/NBK540915/
- BMJ Statistics Notes – “Critical appraisal of clinical trials” (free): concise checklist for RCTs. https://www.bmj.com/content/bmj_stats_notes
- CONSORT 2020 checklist (free PDF): the gold standard for reporting and appraising RCTs. https://www.equator-network.org/reporting-guidelines/consort-2020/
- STROBE guidelines (free PDF): for observational studies. https://www.equator-network.org/reporting-guidelines/strobe/
End of framework. Next: Chapter 19 — Pragmatic trials (already written as Chapter 15 in this framework).