3693 - Clinical Validation of a Deep Learning-Based Auto-Segmentation Model for Organs at Risk in Patients with Thoracic and Breast Cancer: A Prospective, Multicenter, Randomized Clinical Trial in China
Presenter(s)
G. Niu1, Y. Guan1, Y. Zhang2, J. Chen1, X. Wang1, J. Huang3, K. Yang3, J. Z. Cao4, Y. Lu5, and Z. Yuan6; 1Department of Radiation Oncology, CyberKnife Center, and Key Laboratory of Cancer Prevention and Therapy, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer, Tianjin, China, 2Department of Oncology, Institute of Integrative Oncology, Tianjin Union Medical Center, Nankai University School of Medicine, Tianjin, China, 3Cancer Center, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, China, 4Shanxi Cancer Hospital and the Affiliated Cancer Hospital of Shanxi Medical University, Taiyuan, China, 5School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China, 6Department of Radiation Oncology, Tianjin Medical University Cancer Institute and Hospital, National Clinical Research Center for Cancer and Tianjin Key Laboratory of Cancer Prevention and Therapy, Tianjin, China
Purpose/Objective(s): Deep learning (DL)-based artificial intelligence (AI) has rapidly advanced organ at risk (OAR) auto-segmentation in radiotherapy planning. However, previous studies have focused on standalone AI performance or expert-only settings, without adequately considering physician-AI interaction or medical services heterogeneity. We employed a prospective multicenter design to evaluate clinical performance and applicability of a DL-based model (Res-SE net) for AI-assisted delineation of thoracic OARs.
Materials/Methods: Between September 30, 2022, and February 14, 2023, this prospective, multicenter, randomized trial included 500 patients with thoracic and breast cancer from five hospitals in China. Computed tomography images were annotated by 37 physicians using manual, AI, and AI-assisted methods. AI delineation results were generated using Res-SE net, then refined by physicians (AI-assisted delineation). Eleven thoracic OARs were evaluated using performance metrics such as the Dice similarity coefficient (DSC), contouring time, and volumetric revision index (VRI).
Results: We prospectively annotated 2,483 thoracic OAR sets: 993 manual (10,803 OARs), 497 AI (5,403 OARs), and 993 AI-assisted (10,837 OARs) delineations. AI-assisted delineation achieved significantly higher DSC (mean, 0.902) than manual delineation (mean, 0.857) across 11 OARs (p<0.0001), while improving time efficiency by 81.63% and reducing VRI by 1.32-fold (p<0.0001). Regardless of physicians’ centers, expertise, or subjective evaluation of the AI model, AI-assisted delineation consistently outperformed manual delineation in DSC, contouring time, and VRI (p<0.0001), while reducing performance metrics variability across centers and physicians. For OARs with manual DSC<0.8, the AI model demonstrated superior OAR boundary recognition. This boundary recognition improvement was also observed in 12% of less-experienced physicians, with increased manual DSC (p<0.05) after AI model use.
Conclusion: AI-assisted delineation using this model outperforms manual delineation in patients with thoracic and breast cancer, while promoting healthcare equity across centers and physicians. Potential physician-AI interactions were also explored to clarify evolving physician roles in model utilization.