3583 - The Rise of Artificial Intelligence in Radiation Oncology Residency Applications: An Analysis of Applicant Personal Statements, 2015 to 2024

Purpose/Objective(s): Artificial intelligence (AI) has already begun to revolutionize society by simplifying scores of previously labor-intensive tasks, including in academic medicine. Its current role in the residency application process, however, is not well-understood. Thus, the aim of this study was to better understand how radiation oncology (RO) residency applicants have utilized AI in crafting their personal statements over the last decade.

Materials/Methods: We created a database of personal statements submitted by applicants to a single Accreditation Council for Graduate Medical Education (ACGME)-accredited RO residency program via the Electronic Residency Application Service (ERAS) between 2015 and 2024. To estimate the share of each personal statement that was AI-generated, we ran each essay anonymously through zeroGPT. Additionally, each personal statement was scanned for words and phrases that are commonly utilized by AI as defined by Deike (2025). We then used Fisher tests and summary statistics to analyze our results. This study was approved by our Institutional Review Board.

Results: We examined personal statements from 1533 RO residency applications, a sample that represents 73.4% of all RO PGY-2 residency applications over the examined time period. Overall, 6.8% of all personal statements contained = 30% suspected AI content according to zeroGPT, including 12.4% and 4.5% of personal statements in 2023 and 2024, respectively. A higher percentage of suspected AI content was found in the personal statements of men (p = 0.01) and applicants who lacked a higher-level degree (PhD, PharmD, JD, and/or Masters) (p = 0.02). The share of personal statements with = 2, 3, and 4 words commonly used by AI ranged from 3.31 - 5.98%, 0 – 1.25%, and 0 – 0.53% between 2015 and 2023 before rising to 38.8%, 20.7%, and 11.6%, respectively, in 2024 (Table 1). Each of these changes was statistically significant (p < 0.01). Personal statements written by applicants from medical schools outside the US were more likely to contain = 2 and =3 words commonly used by AI than those written by US-based applicants (p < 0.01 for each).

Conclusion: An increasing share of RO residency applicants appear to be utilizing AI to craft their personal statements. Should this trend continue, residency programs will find it more difficult to glean useful knowledge about applicants from these writing samples. A greater emphasis on alternative methods to evaluate an applicant’s ‘fit’ for a particular residency program is warranted.