Forecasting NHS hospital demand under two structural breaks: A multiple-comparison-corrected audit of seven forecasting methods on 15 years of public data
International Journal of Medical Informatics, cilt.219, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 219
- Basım Tarihi: 2026
- Doi Numarası: 10.1016/j.ijmedinf.2026.106579
- Dergi Adı: International Journal of Medical Informatics
- Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, BIOSIS, CINAHL, Compendex, EMBASE, INSPEC, MEDLINE
- Anahtar Kelimeler: Artificial intelligence, Demand forecasting, Health informatics, Machine learning, NHS England
- İstanbul Ticaret Üniversitesi Adresli: Evet
Özet
Background: Machine learning (ML) is increasingly used for hospital demand forecasting, but few published studies report multiple-comparison-corrected statistical tests against competitive baselines, and few characterise the structural-break context in which forecasts are evaluated. Objective: To audit seven AI/ML methods against statistical baselines on 15 years of NHS England activity data, with explicit attention to structural breaks and pandemic confounding. Methods: Monthly A&E activity (n = 189) and 52-quarter KH03 bed occupancy (2010–2026) were obtained from public NHS England releases. Andrews sup-F (q = 2 parameters, 15% trimming) and Bai-Perron breakpoint analyses characterised structural breaks. Seven forecasters were benchmarked on a 12-month chronological hold-out and 12-fold rolling-origin cross-validation, with Diebold-Mariano tests and Bonferroni-Holm + Benjamini-Hochberg correction. A pre-pandemic sensitivity (n = 115) assessed COVID-19 confounding. Detailed methodology is in Appendix A (separate file). Results: 4-h wait failures grew from ∼ 3,700 to ∼ 123,000 per month (∼33-fold). Andrews sup-F = 25.4 placed the demand-side break at 2014–10 (compatible with NHS Five Year Forward View); an independent bed-occupancy break at 2020–09 aligned with NHS COVID-19 reorganisation. On the COVID-inclusive hold-out, only LSTM was significantly worse than naive under Holm correction (p = 0.029); six of seven methods showed no significant difference, but every modern-ML method had a worse point MAPE than the seasonal-naive baseline. Pre-pandemic, only the statistical methods ETS and Prophet significantly beat naive. Under horizon-matched rolling-origin one-step evaluation with Diebold-Mariano testing, no method significantly outperformed the seasonal-naive y[t-12] baseline; the apparent ML advantage held only against the weaker three-year-mean baseline. Conclusion: After horizon-matched, multiple-comparison-corrected evaluation, modern ML showed no reliable advantage over a strict seasonal-naive baseline on monthly system-level NHS data; the statistical methods Prophet and ETS were the only models to beat naive significantly, and only in the pre-pandemic regime. The apparent rolling-origin ML “recovery” reflected an easier one-step task and a weaker baseline rather than a genuine evaluation-scheme effect. The practical message is that complex ML should not be assumed superior for monthly NHS demand planning unless it demonstrably beats a strong seasonal-naive baseline under rolling-origin, multiple-comparison-corrected testing.