3639 - A Study on the Performance of a Deep Learning Gamma Pass Rate (GPR) Prediction Model when the Number of Arcs Used for the Given VMAT Plan are Varied
Presenter(s)
L. Jason1, K. Huang2, F. Motamedi1,3, L. Huang1, J. Liu1, and J. Chang1,4; 1Northwell, New Hyde Park, NY, 2Department of Computer Science and Technology, Kean University, Union, NJ, 3Physics and Astronomy, Hofstra University, Hempstead, NY, 4Department of Physics and Astronomy, Hofstra University, Hempstead, NY
Purpose/Objective(s): In this study, a previously developed in-house deep learning model tasked with predicting Gamma Pass Rates (GPR) of VMAT IMRT radiotherapy plans, is tested further to observe the performance dependance on the quantity of arcs available. If the quantity of arcs provided does significantly influence model performance, then the training process for this model will need to be evaluated as well as the viability of the algorithm for this task.
Materials/Methods: A deep learning (DL) model based on Microsoft’s ResNet architecture was modified to take multiple single channel inputs from DICOM dose plane files. The purpose of this model is to take in 2D dose plane information from each delivered arc of VMAT therapy, and predict the GPR results of IMRT QA. This would be helpful in an Online Adaptive Radiotherapy environment where IMRT QA is impossible to run without removing the patient from the table, and a second check of the deliverability of the plan is necessary. This model had been shown to accurately predict GPR for Prostate/node and Oropharynx plans with a MSE of 0.99 and .90, and a Pearson correlation coefficient of .9 and .99 respectively. This study looks further into the dependencies of this model. With a DL model, features are developed and chosen by the model itself, with only the training data being provided. With the features being unknown, the model must be probed by controlling the provided data and analyzing the resulting changes in performance. This model depends on 2D dose plane inputs with 1 dose plane corresponding to the dose from 1 arc. More arcs in a given plan results in more separated data being provided to the model. Plans were grouped by number of arcs, and the model was tested on performance per group to discover dependencies.
Results: The Prostate/Nodes model decreased in performance with a smaller number of arcs. The performance decreased with the number of fields, showing highest accuracy with 4 fields (MSE = 0.08), and lowest accuracy at 2 fields (MSE = 1.4). For the Oropharynx model, a similar dependency was observed, with 4 fields showing higher accuracy (MSE = 0.40) , and 2 fields showing lower accuracy (MSE = 1.06). All subgroups still show a high correlation coefficient with actual values (>0.95) indicating a very strong correlation with predicting actual values.
Conclusion: The model's performance is influenced by the number of arcs available for input. This may be a training bias due to the number of available plans for training with each set of arcs. For prostate/nodes, it was most common for plans to have 3 or 4 fields. Another possibility is that the increased data from having a higher quantity of arcs to draw from helps with manifesting an accurate prediction. We can conclude that while there is an influence on performance to be investigated, the models still perform as expected for this task.