Main Session
Sep
28
PQA 03 - Digital Health Innovation and Informatics, Patient Safety & Quality, and Radiation and Cancer Biology
2510 - How Many Plans Are Enough? Defining Minimum Cohort Size for Stable Population-Based Plan Quality Scoring
Presenter(s)
Cella Kove, BA - Jefferson Health Sidney Kimmel Cancer Center-Center City, Philadelphia, PA
C. Kove1, S. Marhefka1,2, D. Panchal1, V. Aragam1, Y. Vinogradskiy3, Y. Chen3, W. Choi3, and D. Thomas3; 1Department of Radiation Oncology, Thomas Jefferson University Hospital, Philadelphia, PA, 2Villanova University, Villanova, PA, 3Dept. of Radiation Oncology, Sidney Kimmel Medical College and Comprehensive Cancer Center, Thomas Jefferson University, Philadelphia, PA
Purpose/Objective(s):
Population-derived dose-volume-histogram (DVH) scoring systems are increasingly used to benchmark radiotherapy plan quality; however, the minimum number of historical plans required to generate statistically stable percentile rankings has not been quantitatively defined. This study sought to define the minimum number of historical plans required to achieve statistical stability in a composite DVH-based score for prostate radiotherapy.Materials/Methods:
Clinically delivered prostate Volumetric Modulated Arc Therapy (VMAT) plans with prescribed dose of 70 Gy in 28 fractions and treated at our institution from 2022 to 2026 were analyzed. DVH objectives reflected institutional standards adapted from NRG trial constraints. Achieved DVH metrics were standardized to protocol-specific means (z-scores), sign-adjusted so higher values reflected superior performance, and weighted by institutional priority. Weighted objective scores were summed to generate a composite score, which was converted to a percentile rank, from 0-100, using the empirical cumulative distribution. Stability of plan ranking was assessed by recalculating the composite score of a fixed, median-scoring reference plan across 800 bootstrap samples drawn from progressively larger random samples (N=10–230, step = 10). At each N, the 95% confidence interval (CI) of the reference score was computed using the 800 bootstrapped samples; stability was then defined as =2.5% relative reduction in CI width. Distributional convergence was independently evaluated using the Kolmogorov–Smirnov distance test on the full-cohort and bootstrap score distributions.Results:
238 plans that were generated between 2022-2026 were evaluated. Confidence interval width of the reference plan score decreased with increasing sample size and plateaued at approximately N=200, where the relative reduction in CI width fell below 2.5%. CI width decreased from 1.237 at N=50 to 0.599 at N=200 (at N=230, CI width = 0.552). Kolmogorov–Smirnov distance similarly stabilized near N=200 (median 0.056, 95% CI [0.030-0.104]), confirming distributional convergence.Conclusion:
For the first time, this study establishes that ~ 200 historical plans are required to achieve stable percentile-based plan scoring in prostate radiotherapy. Protocols with a low volume of plans or newly implemented treatment pathways may lack sufficient historical data for reliable percentile benchmarking. While thresholds vary by disease site, this method provides a reproducible framework to define a minimum cohort requirement for population-based plan scoring.