2410 - Large Language Model Augmented Topic Modeling of a Decade of Radiation Oncology Research Abstracts

Purpose/Objective(s): IJROBP publishes nearly 2,000 abstracts annually, making it challenging to systematically track field-wide trends. We hypothesize that LLMs can validate and label topic-modeled themes from ASTRO abstracts, enabling reproducible tracking of research priorities to inform future research in radiation oncology.

Materials/Methods: We analyzed 4,277 abstracts from IJROBP (2013-2023). BERTopic with PubMedBERT embeddings, UMAP reduction, and HDBSCAN clustering yielded 100 topics. Outlier optimization reduced unassigned abstracts from 18.6% to 6.9%. A two-phase LLM validation pipeline followed: Phase 1 used c-TF-IDF similarity with GPT-4o-mini to evaluate merges (99 final topics); Phase 2 scored topic fit (1-5), producing 427 reassignments improving coherence (c_v: 0.630 to 0.641). Labeling used GPT-3.5 Turbo and Gemini 2.5 Flash.

Results: The pipeline identified 99 clusters (Table 1). Growing clusters were smaller topics with recent surges, notably oligometastatic SBRT (28 abstracts in 2022 vs. average of 5/year) and a COVID-19 cluster emerging post-2019. Declining clusters like Hodgkin lymphoma proton therapy reflect waning interest in established topics.

Conclusion: LLM-augmented topic modeling identified and validated distinct themes across a decade of ASTRO abstracts. The field is shifting toward hypofractionation, stereotactic techniques, immunotherapy combinations, and FLASH, while conventional fractionation and older paradigms decline. This framework helps scientists track publication trends and could inform strategic research planning.