3709 - Validation of a High-Dimensional Machine Learning Model to Predict Hospital Mortality amongst Patients Requiring Urgent Radiation
Presenter(s)
E. M. Qiao1, A. S. Qian2, B. J. Jacobs3, A. J. Lui4, M. E. Orr5, A. C. Puett4, A. Dornisch4, K. Guram6, J. T. Butler7, A. B. Hopper4, S. Kim8, P. Riviere9, M. D. Tibbs4, and J. D. Murphy10; 1Department of Radiation Medicine and Applied Sciences, UC San Diego Health, La Jolla, CA, 2University of California, San Diego, San Diego, CA, 3UC San Diego, La Jolla, CA, 4Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, 5Center for Health Equity Education and Research, University of California, San Diego, La Jolla, CA, 6University of California, San Diego Moores Cancer Center, La Jolla, CA, 7Oregon Health and Science University, Portland, OR, United States, 8UC San Diego, Moores Cancer Center, Department of Radiation Medicine and Applied Sciences, La Jolla, CA, 9University of Washington Department of Radiation Oncology, Seattle, WA, 10Department of Radiation Medicine and Applied Sciences, UC San Diego, La Jolla, CA
Purpose/Objective(s): The decision to deliver urgent radiation in the inpatient setting relies on estimated patient survival. Yet, among inpatient cancer patients with indications for radiation we lack effective validated models to predict survival. We previously trained a high-dimensional machine learning (ML) model – the Cancer Frailty Assessment Tool (cFAST) – that showed high accuracy for identifying cancer patients at risk of in-hospital mortality. Here, we externally validate cFAST’s ability to predict in-hospital mortality amongst hospitalized cancer patients with indications for urgent radiation within a tertiary cancer center.
Materials/Methods: We developed cFAST from the National Emergency Department Sample (NEDS) years 2016-2018 utilizing extreme gradient boosted models to predict in-hospital mortality. Model covariates included patient demographics and International Classification of Diseases, version 10 (ICD-10) diagnosis codes recorded during the ED visit. With this current external validation, we utilized a prospectively maintained cohort of hospitalized cancer patients who required urgent radiation from February 2024-April 2024 at a single tertiary cancer center. Manual electronic health record (EHR) review identified demographics and ICD-10 diagnosis codes from each hospital encounter. Area under the curve (AUC) evaluated model performance.
Results: Our validation cohort included 39 patients with an indication for urgent radiation. The most common cancer primaries were lung (17.6%), prostate (10.3%), and bladder (10.3%), with 33 (84.6%) metastatic patients. The median age was 63 (interquartile range 54-71), with 12 female (30.8%) and 27 male (69.2%) patients. The most common indications for urgent radiation were brain metastases (28.2%), pain (20.5%), and spinal symptoms (17.9%). There were 5 in-hospital mortality events (12.8%). The AUC of cFAST on this validation cohort was 0.76, compared with the prior benchmark training cohort AUC of 0.92.
Conclusion: External validation of cFAST showed promising discriminatory power for hospital mortality within a limited sample of cancer patients. The predictive capacity of cFAST within this validation cohort is lower than the initial training cohort, highlighting potential opportunities for model calibration or refinement within new cohorts. However, ML models such as cFAST have clear potential to help risk-stratify patients receiving radiation in the real-world setting. Future integration into EHRs will enable more expansive implementation into clinical workflows.