Main Session
Sep
29
PQA 07 - Head and Neck Cancer, Lung Cancer/Thoracic Malignancies, and Nursing and Supportive Care
Presenter(s)
Penghao Gao, - Shandong Cancer Hospital & Institute, Jinan, Shandong Provinc
P. Gao1, Y. An1, G. Zhang1, Y. Sun2, and L. Wang1,2; 1Artificial Intelligence Laboratory, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, China, 2Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University and Shandong Academy of Medical Sciences, Ji'nan, China
Purpose/Objective(s):
Currently, Virtual Reality (VR) and Augmented Reality (AR) technologies are advancing rapidly, yet its application in radiotherapy remains largely confined to educational training and medical model demonstrations, with limited integration into actual treatment scenarios. This study utilizes the HoloLens 2 device to construct a high-fidelity, interactive, and immersive VR/AR-based radiotherapy workflow environment. This platform was designed to facilitate patient education to achieve precision, intelligence, and visualization in treatment delivery.Materials/Methods:
In Unity3D, we built relevant scenes for the entire radiotherapy process, including the patient login interface, information desk, doctor's workstation, simulated positioning room, and radiotherapy area. Three-dimensional models of relevant images, treatment planning target volumes, and radiotherapy equipment were imported into these scenes. The complete workflow was integrated and deployed to the HoloLens 2 using Visual Studio, facilitating interactive scene operations and external communication with a PC. Integrated Natural Language Processing (NLP) capabilities enabled real-time, voice-based question-and-answer interactions with participants throughout the workflow.Results:
The entire process enables functions such as targeted educational video playback, dynamic synchronization of 4D CT models with respiratory motion amplitude, intelligent positioning of treatment plans, and dynamic adjustment of treatment beams. Positioning accuracy reaches the millimeter level. Following the subject's voice, the speech recognition results are transmitted to the workstation within 2 seconds. The self-developed NLP question-answering model then completes response within 6 seconds, with the results subsequently returned to the HoloLens 2 device for both audio playback and text display.Conclusion:
Immersive intelligent education systems can provide personalized education based on patients' treatment plans, allowing patients to intuitively experience the entire treatment process, increasing patient cooperation, fundamentally improving radiotherapy effects, and deeply integrating VR/AR into radiotherapy.