30, 18, 4
Judge a prediction model by its performance in the window in which the decision is taken, not by its best reported figure. A model needing the last 24 hours before decannulation to predict decannulation is describing, not predicting.
A model built on the RESP score alone
Sivakumar S, Fan L, Cao M, Brown P, Kim BS, Whitman GJ, Cho S, et al. Artif Organs 2026. DOI 10.1111/aor.70198
Machine learning models (Random Forest, CatBoost, AdaBoost, XGBoost) built on EMR time-series across four windows: pre-ECMO, first 24 h, last 24 h before decannulation, and the full run. 80/20 train-test splits; SHAP feature importance.
119 patients, 62 events, 80/20 split, four algorithm families compared — roughly 12 events in the test set. No external validation. The abstract presents a WEANING model but reports only mortality at discharge as its outcome statistic, so the endpoint definition is not recoverable [VERIFICATION REQUIRED]. The abstract also prints 'median age = 48%'. Abstract only.
AUROC by model: RESP alone 0.815; pre-ECMO 0.715; first 24 hours 0.515; last 24 hours before decannulation 0.881; full ECMO run 0.904. The authors conclude high-granularity models outperform RESP-based models. READ BY DECISION WINDOW the conclusion reverses: the two best-performing models require the outcome to have already occurred, while the two windows in which a prediction would be wanted gave 0.715 and 0.515 against a 4-variable pre-cannulation score at 0.815.
119 adults (18 years and over) on venovenous ECMO at Johns Hopkins Hospital 2016-2024; 59.6 percent female; 62 (52.1 percent) mortality at discharge