3708 - Large Scale Breast Subset Analysis of Practice Patterns Emerging from an Automated Real-Time Evaluation of a Commercial Artificial Intelligence Auto-Contouring Solution
Presenter(s)
P. Dubrowski1, R. Prashad1, M. J. Kim2, Y. Yang3, and D. H. Hristov1; 1Department of Radiation Oncology, Stanford University School of Medicine, Palo Alto, CA, 2University of California, San Francisco, San Francisco, CA, 3Department of Radiation Oncology, Stanford University, Stanford, CA
Purpose/Objective(s): Segmentation by Artificial Intelligence (AI) has been shown to greatly improve planning efficiency. Upon implementation of a commercial AI auto-contouring tool at our multi-site academic network, we developed an unsupervised software framework to automatically evaluate segmentation performance for every patient. This study explores emerging practice patterns in breast patient data over one year, determining whether disease site specialists edit structures more frequently than others, whether planning target edits impact organ at risk (OAR) dose, and whether edits occur mainly when contours are essential to the planning process. Additionally, we examine potential interventions for increased practice standardization.
Materials/Methods: An evaluation tool was developed for autonomous comparison of clinically approved and initial AI structures geometrically and dosimetrically. Over one year, we analyzed over 900 patient plans and 15,500 structures; breast was the most frequent anatomical site and was used in this subset analysis (207 plans and 3242 structures). AI breast contours include regional lymph nodes (LNs), which can direct clinical planning. A Relative Mean Clinical Dose (rMCD) > 80% was used as a surrogate to infer plan coverage intent for planning target contours (PTCs). To determine if a contour was modified, a combination of Dice Score and Relative Volume Difference metrics were used. Two of eight physicians were designated breast-experts, accounting for 68% of all records.
Results: PTCs were modified more frequently than non-PTCs (53% vs 8%, P< .0001). Non-PTCs most modified were spine and level II LNs; heart and esophagus were minimally edited. 40% of all LNs were modified, with left IMN LNs edited 100%. Edits to PTCs moderately reduced heart and lung doses; heart rMCD decreased by approximately 1% for any left LNs edited (P<0.05). All LN edits resulted in reduced volume. Experts modified PTCs more frequently: 60% vs 36% (P< .0001) and edited left-sided LNs most frequently (P<0.05), while non-PTCs and breast contours were edited at similar rates.
Conclusion: Higher rates of editing in PTCs indicate scrutiny when contours are important to planning (especially LN targets); spine, level II LN were frequently edited regardless of dose suggesting anatomical drivers; heart and esophagus were unedited and may not require much attention in planning. LN PTC edits trended with moderate reductions in lung and heart OAR doses; left-sided LN edits were significantly correlated with reduced mean heart dose. Experts modified PTCs, especially left-sided LNs, significantly more suggesting higher comfort with LN anatomy or intent to reduce dose to OARs. This points to a specific area for improvement to ensure standardization of patient care among disease site physicians at our clinics.