116 - Perceptions, Behavioral Impact and Vulnerabilities of the ACGME Resident Survey in Radiation Oncology: A Mixed-Methods Analysis of Radiation Oncology Program Leadership
Presenter(s)
S. M. McVorran1, E. Collins2, E. M. M. Janowski3, S. L. Cooper4, C. Henson5, O. Adelugba2, D. O'Connell2, K. Postlewait6, and A. H. Masters7; 1Department of Radiation Oncology, University of Washington/Fred Hutch Cancer Center, Seattle, WA, 2University Of Louisville School of Medicine, Louisville, KY, 3Department of Radiation Oncology, University of Virginia, Charlottesville, VA, 4Department of Radiation Oncology, Medical University of South Carolina, Charleston, SC, 5University of Oklahoma Health Sciences Center, Oklahoma City, OK, 6Oklahoma University, Oklahoma City, OK, 7Department of Radiation Oncology, University of Louisville, Louisville, KY
Purpose/Objective(s): The ACGME Resident Survey plays a central role in accreditation and compliance across graduate medical education, assessing resident experience and identifying areas for improvement. This study evaluated radiation oncology program leaders’ perceptions of the survey’s accuracy and fairness, its influence on programmatic decision-making and leadership well-being, and qualitative experiences regarding survey misuse and potential safeguards.
Materials/Methods: A national mixed-methods survey of U.S. radiation oncology program leadership (program directors, associate program directors, and program coordinators) was conducted using a combination of 5-point Likert-scale items, other ordinal response options (including yes/no/unsure and frequency scales), and open-ended questions. Quantitative analysis included descriptive statistics and subgroup comparisons with two-tailed testing (p=0.05). As the study was descriptive and not powered for small-N comparisons, emphasis was placed on effect sizes and prevalence. Qualitative responses were thematically analyzed to characterize perception, behavioral impact, leadership burden, and recommended reforms. The data collection web application survey generated 93 complete responses.
Results: Respondents reported low confidence that the survey reflects program reality (mean˜2.9/5) or provides a fair assessment (mean˜2.5/5), while strongly agreeing that it does not capture program complexity (mean˜4.3/5). No statistically significant differences by program size were observed across most measures. However, the comparison assessing influence on programmatic decisions yielded p=0.05, and qualitative themes suggested greater vulnerability in small programs. Survey-related concerns influenced programmatic decision-making in 82% of respondents (42% frequently; 40% occasionally). While some respondents described constructive quality improvement initiatives, qualitative themes more often reflected defensive adaptations including reduced rigor, hesitancy to give constructive feedback, and avoidance of resident accountability. The leadership burden of the survey was substantial (burnout mean 3.8/5). Qualitative themes included perceived misrepresentation of program conditions, concerns regarding retaliatory or coordinated negative reporting leading to citations, site visits, and emotional distress. Suggested improvements included contextualizing results, clarifying items, adding comment fields, enabling appeals with submission of objective data, and providing ACGME-led resident education on survey use.
Conclusion: Radiation oncology leaders view the ACGME Resident Survey as insufficiently nuanced and sometimes distortionary, with the potential to reduce educational rigor and increase leadership burden. Greater contextualization, clearer items, and less punitive interpretation may improve fairness and support constructive program improvement.