2646 - Quantitative Risk Assessment in Radiation Oncology via LLM-Powered Root Cause Analysis of Incident Reports
Presenter(s)
Y. Wang1, S. Nejad-Davarani2, E. L. Bossart3, M. T. Studenski4, M. De Ornelas5, and Y. Yang5; 1University of Miami - Sylvester Cancer Center, Coral Gables, FL, 2Department of Radiation Oncology, Sylvester Comprehensive Cancer Center, Miller School of Medicine, University of Miami, Miami, FL, 3Department of Radiation Oncology, University of Miami School of Medicine, Miami, FL, 4Department of Radiation Oncology, University of Miami/Sylvester Comprehensive Cancer Center, Miami, FL, 5University of Miami, Miami, FL, United States
Purpose/Objective(s): Root Cause Analysis (RCA) is resource-intensive and prone to subjective bias. While Radiation Oncology Incident Learning System (RO-ILS) collects vast data, extracting insights from unstructured narratives remains a bottleneck. Large language models (LLMs) offer powerful reasoning capabilities that convert narratives into structured data, enabling quantitative risk assessment to identify systemic risks and vulnerabilities.
Materials/Methods: We developed a pipeline using an LLM to perform automated RCA on 254 institutional safety incidents. The LLM classified each incident into the UK Health Security Agency (UKHSA) National Patient Safety Radiotherapy Event Taxonomy, including Contributory Factors and Pathways of Failure. Each incident was assigned a severity score based on TG-100. Quantitative analyses included: Pareto analysis for frequency profiling; Box plots and One-way Analysis of Variance (ANOVA) to assess severity distribution; Ordinal Logistic Regression (OLR) to identify high-risk predictors; and Association Rule Mining (ARM) to visualize latent failure pathways.
Results: Pareto analysis identified "Treatment Delivery" and "Human Behavior" as primary frequency drivers. Significant severity differences were observed across Pathway steps (p < 0.05). Co-occurrence analysis indicated "Procedural" and "Individual factors" were the most prevalent contributory factor categories, with "Department Leadership" and "Physicians" as the most common responsible parties. High-frequency contributory factors included "Process Design" (n=142) and "Slips and Lapses" (n=111). The OLR model identified "Inadequate Leadership" (p < 0.001) and procedural "Violation" (p < 0.001) as significant high-risk predictors of incident severity.
Conclusion: We demonstrated an LLM-powered quantitative risk assessment by converting unstructured incident reports into a structured dataset. This approach enables the identification of key drivers of event severity and the mapping of complex, systemic failure pathways. This methodology paves the way for a new generation of intelligent safety systems to improve the quality and safety of care.