3048 - Large Language Model-Driven Zero-Shot Style Adjustment for Breast Cancer Target Delineation: Based on Instruction Understanding and Anatomical Relationships
Presenter(s)
J. Lan1,2, D. Gao3, X. Sun1, R. Zhong4, X. Wu1,2, X. Chen1, L. LIU1,2, L. Jia5, W. Zhang3, and J. Jing1; 1Institute of Breast Health Medicine, State Key Laboratory of Biotherapy, West China Hospital, Sichuan University and Collaborative Innovation Center, Chengdu, Sichuan, China, 2Division of Head and Neck Tumor Multimodality Treatment, Cancer Center, Institute of Breast Health Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan, China, 3Shanghai United Imaging Healthcare Co., Ltd., Shanghai, China, 4Department of Radiotherapy Physics & Technology, West China Hospital, Sichuan University, Chengdu, Sichuan, China, 5Shenzhen United Imaging Healthcare Co., Ltd., Shenzhen, Guangdong, China
Purpose/Objective(s):
Significant stylistic variations in breast cancer (BC) target delineation across radiation oncologists (ROs) limit model generalization. Traditional solutions require collecting extensive specific annotated data for model retraining—a time-consuming process that burdens ROs. Moreover, as precision radiotherapy advances and guidelines evolve, ROs repeatedly provide new annotations for model updates, exacerbating repetitive labor. Hence, a method that eliminates large-scale annotation, interprets ROs instructions, and utilizes anatomical relationships for adaptive delineation style adjustment is urgently needed.Materials/Methods:
We propose a zero-shot adaptive style adjustment strategy for BC target delineation based on a large language model (LLM), called Qwen. This strategy firstly relies on a foundational BC target model and the muscle and bone models associated with each BC target region. On this basis, LLM is employed to interpret ROs' instructional modifications to the target and map them into specific post-processing operations (e.g., expansion, contraction, inclusion scope adjustment). By invoking predefined functions and integrating the anatomical spatial relationships between the targets and surrounding normal tissues, zero-shot precision adjustment of target delineation style is achieved. The entire process requires neither model fine-tuning nor additional annotated data.Results:
As the core foundation of the proposed strategy, the BC target model has been refined according to the latest clinical guidelines, with axillary and supraclavicular lymph nodes subdivided into axillary levels I, II, III and supraclavicular levels I and II, respectively, enhancing its adaptability for multi-center. The average Dice of the target foundation model is 0.85, while that of the muscle and bone models reaches 0.95 in average. In addition, we have developed an image visualization module. It features an embedded tool list and supports direct invocation of a LLM dialog box. Upon receiving modification instructions, the LLM completes instruction parsing within 10 seconds, outputs procedural spatial operation parameters, and invokes predefined functions to automatically adjust the target delineation style. Preliminary in-hospital test results demonstrate that, based on the target generated by the foundation model, the LLM’s interpretation of modification instructions from different ROs can effectively adjust the target delineation to align with individual ROs styles.Conclusion:
We have preliminarily validated the clinical feasibility of using LLM to convey ROs modification instructions for zero-shot adaptation to different BC target styles. This approach provides a new pathway to enhance the generalization capability and clinical utility of target delineation models.