The ICU morning sign-out
You’re at the sign-out board. “Our newest sepsis trial randomized 200 patients to early high-volume fluid vs conservative. Mortality 28% vs 34%. p = 0.04.”
Your colleague nods. “Better fluids. I’ll change the protocol.”
Before you do, ask: Was this an RCT? Or an observational analysis dressed in RCT clothes? The distinction matters because the methods you use to appraise the paper, and the assumptions you make, differ completely.
The two design families
Randomized controlled trials (RCTs)
In an RCT, the investigator decides the exposure. Patients are allocated by a chance mechanism — a computer-generated sequence, sealed envelopes, or a web randomizer — before outcome data are collected. The key feature is prospective allocation.
Why does chance matter? It balances both known and unknown confounders across arms, on average. If you randomize well, the only systematic difference between groups at baseline is the intervention. That is why RCTs are the gold standard for causal questions: “Does this intervention cause this outcome?”
Observational studies
In observational research, the exposure happens naturally. Physicians decide who gets the new protocol, or patients choose (or fail to choose) a treatment. The investigator observes rather than assigns.
Three main subtypes you’ll encounter:
Subtype | How subjects enter | Typical clinical question |
Cohort | A group is followed forward in time, exposed or not exposed, and outcomes are recorded. | “Does chronic corticosteroid use increase VAP risk in ICU patients?” |
Case-control | Starts with outcomes (cases) and looks backward for exposures. | “Among patients with AKI stage 3, what proportion received contrast imaging in the prior week?” |
Cross-sectional | A snapshot at a single time point. Exposure and outcome measured simultaneously. | “What proportion of ventilated patients receive subglottic secretion drainage, and what’s the VAP rate?” |
Why the distinction matters for appraisal
Question | Best design | Why |
“Does X cause Y?” | RCT (if ethical and feasible) | Randomization balances confounders; temporal sequence is clear. |
“How many get X and what happens?” | Observational cohort | You can follow patients forward; incidence can be calculated. |
“Is X associated with Y in a population already receiving X?” | Case-control or cross-sectional | Efficient when outcome is rare; can estimate odds or prevalence. |
If a paper claims causality from an observational design, you must scrutinize confounding far more heavily. If an RCT claims only association, check whether the randomization was properly executed and whether the analysis respects the intention-to-treat principle.
The anatomy of a well-reported RCT
The CONSORT 2020 statement gives a checklist, but the essential elements you can verify in the methods section are:
- Sequence generation: Was the random allocation truly unpredictable? (e.g., computer-generated permuted blocks with random block sizes, not alternation.)
- Allocation concealment: Could someone enrolling a patient predict or influence the assignment? (Sealed opaque envelopes, central web randomizer, pharmacy-controlled.)
- Blinding: Who was blinded? Participants, clinicians, outcome assessors, data analysts, or all of the above? If unblinded, what steps were taken to reduce performance and detection bias?
- Baseline balance: Are the key prognostic factors (age, APACHE score, comorbidities) balanced between arms? (Usually shown in a table.)
- Intention-to-treat (ITT) analysis: Did the analysis include all randomized patients in the groups to which they were randomized, regardless of protocol adherence? Per-protocol analyses answer a different question.
- Sample size justification: Was the study powered to detect a clinically important difference? Look for the assumed effect size, event rate, alpha, and power.
The anatomy of a well-reported observational study
Observational papers lack randomization, so you must check for:
- Clearly defined population: Inclusion/exclusion criteria stated; how the cohort was assembled.
- Exposure definition: How was “receiving X” measured? Was it a prescription, a protocol order, a biomarker? Was timing clarified?
- Outcome ascertainment: How was the outcome determined? Chart review? Administrative codes? Mortality from death certificates? What about loss to follow-up?
- Confounder measurement and adjustment: Were potential confounders (age, severity of illness, comorbidities) measured before the exposure? Were they adjusted for in the analysis (multivariable regression, propensity scores, inverse probability weighting)? Were the adjusted and unadjusted results both presented?
- Temporal sequence: Was exposure measured before outcome? In cohort studies, this is essential; in case-controls, the exposure must precede the outcome.
- Sensitivity analyses: Did the authors test the robustness of their findings? (e.g., different confounder sets, alternative definitions, exclusion of early deaths.)
A clinical example: fluid strategy in sepsis
Study type | Example headline | What to check |
RCT | “The FEAST trial: early goal-directed fluid resuscitation in sepsis reduced 28‑day mortality.” | Was there true randomization? Was the fluid strategy actually different between arms (crystalloid vs colloid, volume goal)? Was mortality assessed blindly? |
Observational cohort | “ICU patients receiving conservative fluid strategy had lower VAP rates.” | Was fluid strategy assigned by physician preference (confounding by indication)? Were APACHE scores balanced? Was VAP diagnosed by standardized criteria? |
Your appraisal checklist for design
When you open a paper, apply these four questions before reading the results:
- What design is this? — RCT, cohort, case-control, cross-sectional? Is the stated aim matched to the design?
- Allocation method (RCT only): Was randomization genuine, or was there “quasi-randomization” (alternation, hospital number, day of week)?
- Comparability (observational only): Are the exposed and unexposed groups comparable at baseline on key prognostic factors? If not, how did the authors address it?
- Does the analysis respect the design? — ITT for RCTs; confounder adjustment for observational studies; appropriate survival methods for time-to-event data.
Go deeper
- EQUATOR Network – CONSORT 2020 explanation and elaboration (free PDF): walks through each checklist item with real examples. https://www.equator-network.org/reporting-guidelines/consort-2020/
- StatPearls – “Study Designs in Epidemiology” (NBK430685): free, covers RCT, cohort, case-control, cross-sectional, and when to use each. https://www.ncbi.nlm.nih.gov/books/NBK430685/
- BMJ Statistics Notes – “Studies of observation” (Altman & Bland): concise appraisal points for non-RCT designs. https://www.bmj.com/content/bmj_stats_notes
Next: Chapter 3 — Randomization, allocation concealment & blinding