2697 - A Large Language Model-Augmented Multimodal Framework for Predicting Pain Relief Outcomes Following Stereotactic Body Radiotherapy (SBRT) in Spinal Metastases: Integrating Clinical Factors and Imaging Reports
Presenter(s)
K. Zhang1, X. Ye2, G. A. Szalkowski3, Z. Yang4, C. Chuang4, L. Wang5, L. Liu6, S. G. Soltys4, E. L. Pollom4, E. Rahimy4, J. Byun6, D. Park7, Y. Hori7, F. Lam7, D. Reesh7, S. D. Chang7, G. Li7, M. Hayden8, M. Kazemimoghadam9, Q. Wang9, M. Chen9, H. Jiang10, W. Lu9, and X. Gu11; 1Department of Radiation Oncology, Stanford University, s, CA, 2Department of Radiation Oncology, The First Affiliated Hospital, Zhejiang University School of Medicine, Hangzhou, China, 3Georgia Institute of Technology, Atlanta, GA, 4Department of Radiation Oncology, Stanford University School of Medicine, Stanford, CA, 5Department of Radiation Oncology, Stanford University School of Medicine, Palo Alto, CA, 6Department of Radiation Oncology, Stanford University, Stanford, CA, 7Department of Neurosurgery, Stanford University School of Medicine, Stanford, CA, 8Department of Neurosurgery, Stanford University, Stanford, CA, 9Medical Artificial Intelligence and Automation (MAIA) Lab, Department of Radiation Oncology, UT Southwestern Medical Center, Dallas, TX, 10Department of Radiation Oncology, The University of Texas Southwestern Medical Center, Dallas, TX, 11Stanford University Department of Radiation Oncology, Palo Alto, CA
Purpose/Objective(s):
Accurate prediction of pain relief following spinal stereotactic body radiotherapy (SBRT) is essential for assessing treatment effectiveness and optimizing patient care; however, this remains challenging due to the complex and multifactorial nature of pain. We hypothesis that the multimodal deep-learning framework, which integrates augmented clinical factors and imaging features extracted from reports using large language models (LLMs) can lead to accurate prediction of pain relief outcomes.Materials/Methods:
We retrospectively collected 160 spine SBRT cases with spinal metastases from our institutional frameless robotic radiosurgery system database partitioned into 104 cases for training, 26 for validation and 30 for testing. Each case included two modalities of data: clinical factors and imaging reports. The proposed framework comprises three key components: (1) a data augmentation strategy to encode clinical features such as tumor type, metastasis location, and pain severity; (2) an LLM-driven analyzer (e.g., ChatGPT-4o) to extract high-dimensional semantic embeddings directly from narrative and impression sections of imaging reports; and (3) a cross-attention-based transformer classifier that dynamically fuses clinical data and imaging features to capture interdependencies and enhance predictive accuracy. By unifying these components into an end-to-end workflow, our framework takes clinical factors and image reports to predict the outcome of pain relief conditions.Results:
The proposed method achieved an accuracy of 85.15% and an area under the curve (AUC) of 0.88 (as shown in Table I). Additionally, we conducted an ablation study using only the clinical factors as a single modality. The preliminary results indicated enhanced performance with multi-modal data, which surpassed the 80.95% accuracy achieved using the single modality.Conclusion:
Our study introduces a novel LLM-augmented multimodal framework that integrates clinical factors and imaging reports to predict pain relief outcomes following SBRT for spinal metastases. By leveraging a cross-attention transformer to fuse structured clinical data with semantic embeddings extracted from imaging narratives, our multimodal framework outperforming the single-modal approach. These results underscore the value of combining advanced LLM with multimodal data to enhance predictive precision in oncology workflows. Abstract 2697 - Table 1: Evaluation of methods based on multi-modal inputs in single-modal (clinical factors) inputs| Accuracy | Sensitivity | Specificity | AUC | |
| Multi-Modal | 85.15% | 93.02% | 79.31% | 0.88 |
| Single-Modal | 80.95% | 85.00% | 73.91% | 0.81 |