Presenter(s)
S. Han-Oh1, R. Anderson1, N. Ascencio1, L. Bell1, M. Duhon1, B. Floreza2, R. Genego1, K. Gress1, A. N. Halthore2, E. Huang1, X. Jia1, J. Merrill1, A. Narang1, A. N. Souranis1, P. Thomas1, J. L. Wright3, J. Yu2, and A. N. Viswanathan1; 1Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins Medicine, Baltimore, MD, 2Department of Radiation Oncology and Molecular Radiation Sciences, Johns Hopkins Medicine, Washington, DC, 3University of North Carolina at Chapel Hill, Chapel Hill, NC
Purpose/Objective(s): To evaluate the impact of the ASTRO Accreditation Program for Excellence (APEx) on a multi-facility institution across three accreditation cycles—pre-accreditation (C1), first accreditation cycle (C2), and second accreditation cycle (C3)—using data from an incident learning system (ILS).
Materials/Methods: This five-facility institution achieved APEx accreditation in May of 2017 and renewed in October of 2022 on first attempt. A total of 5,514 ILS events from 2015–2025 were mapped to 55 APEx evidence indicators (EIs) and grouped into the three accreditation cycles (C1-C3). Patient volume-adjusted ILS event rates per 1,000 patients (IER) were calculated for each cycle to assess site-specific and EI-specific changes. IER ratios of later to earlier cycles were used to quantify the effects of initial and renewal accreditation, with ratios <1 indicating improvement. Inter-facility variation was evaluated using the Friedman test, where higher ?² indicated larger differences and Kendall’s W >0.3 suggested moderate to strong variation.
Results: Across all facilities, total IER decreased from C1 (246.5) to C2 (67.6; ratio C2 to C1 = 0.3) and increased in C3 (217.5), but C3 still remained slightly below C1 (ratio C3 to C1= 0.9). Average IER ratios were 0.9±0.8 (median 0.5, interquartile range IQR 0.2-1.5) for C2 to C1, 1.8±2.2 (median 0.8, IQR 0.5-2.5) for C3 to C1, and 3.9±5.6 (median 2.0, IQR 0.9-4.6) for C3 to C2. EIs showing significant changes across all cycles (p<0.05) were standard operating procedure (SOP; IER = 89.7, 36.6, 143.4 for C1, C2 and C3, respectively), informed consent (50.6, 0.4, 11.5), simulation directive (48.6, 2.6, 21.3), patient timeout (26.1, 2.7, 4.2), treatment planning directive (5.1, 2.1, 12.2), and staff on-boarding (2.3, 0.2, 1.1). Prescription (9.4, 5.1, 6.3) showed significant changes only from C1 to C2 or C3, and treatment plan documentation (8.6, 11.9, 10.3) only from C1 to C2. The Friedman test indicated small-to-moderate inter-facility variations across all three cycles (p<0.05): ?²=23.1, W=0.2 for C1, ?²=11.0, W=0.1 for C2, ?²=34.4, W=0.3 for C3.
Conclusion: This study demonstrates a quantitative evaluation of APEx accreditation effects using longitudinal ILS data across a multi-facility institution. SOP-related ILS events markedly increased during the renewal cycle, suggesting a need for strengthened SOP governance, such as SOP change management, staff training, and compliance monitoring, to sustain initial improvements. Significant inter-facility variation suggests that emphasis on individual APEx EIs can be tailored to the specific workflows and challenges of each facility. These findings highlight the value of a data-driven evaluation of accreditation outcomes, offering both high-level and detailed insights across cycles and facilities. Such analyses can support more effective use of accreditation to drive sustainable, facility-specific improvements in safety and quality.