2576 - Development of a Post-Radiation "Safety Net": A Machine Learning (ML) Approach to Predicting Severe Adverse Events (AE) Using the SEER Database

Purpose/Objective(s): To detect acute severe radiation therapy (RT) complications and support value-based care initiatives such as the Radiation Oncology Case Rate program, we propose a machine learning "safety net" for clinical monitoring prioritization. We hypothesized that with accessible registry data, the US SEER dataset can effectively be used to evaluate ML classifiers predicting "early mortality" (<3 months post-RT) as a proxy for severe acute RT-associated AEs.

Materials/Methods: We retrospectively analyzed 235,035 RT-treated patients from the SEER database (2010-2022). The primary outcome was early mortality (<3 months post-RT). We defined a 9-variable feature set across medical, socioeconomic (e.g., median income, rurality, race, sex, marital status), and metadata domains, prioritizing patient-reportable inputs. Data were split into training and held-out validation cohorts. We framed two complementary prediction tasks: classification (Random Forest, Gradient Boosting) and survival analysis (Cox Proportional Hazards). Performance was evaluated using discrimination (C-index) and calibration (Brier score). Finally, to assess generalizability versus specificity, we compared a unified global model (trained on all data) against site-specific models (trained on individual sites), evaluating both on the same site-specific validation sets (lung, breast, prostate).

Results: Early mortality prevalence was 4%, creating significant class imbalance. Despite this, the Gradient Boosting classifier achieved 0.92 recall, successfully flagging most high-risk cases. Precision was 0.10, an acceptable trade-off for a safety-net tool prioritizing sensitivity to minimize missed interventions. In survival analysis, the global Cox model achieved a 0.80 C-index, though discrimination was sensitive to site heterogeneity (average within-site C-index: 0.76). Site-specific models partially recovered this performance (average within-site C-index: 0.78). Across all models, age and socioeconomic factors—specifically income and marital status—emerged as significant, independent predictors of high-risk outcomes.

Conclusion: This study demonstrates that registry data can effectively train highly sensitive machine learning models to predict 3-month post-RT mortality. The high recall (0.92) validates the framework's utility as a screening tool, while socioeconomic factors remain as key drivers that should be integrated into risk assessment. To move beyond mortality as a proxy, next steps will leverage Statistics Canada data to capture actual RT-related AEs and integrate richer, granular socioeconomic features. Ultimately, multicenter validation utilizing independent, granular electronic medical record datasets is warranted to refine this “safety net” framework for clinical deployment.