2461 - Autonomous Dose and Dose-Rate Optimization for Proton Conformal FLASH Radiotherapy Using Deep Reinforcement Learning

Purpose/Objective(s):

Proton conformal FLASH radiotherapy must simultaneously satisfy clinical dose goals and deliver ultra-high-dose-rates (UHDR) under machine constraints. This highly coupled optimization problem is currently addressed through manual trial-and-error within the treatment planning system (TPS), limiting plan optimality and hindering clinical translation. This study develops a Deep Reinforcement Learning (DRL)–based planning agent to autonomously optimize dose quality and UHDR delivery for head-and-neck (HN) reirradiation.

Materials/Methods:

15 HN reirradiation patients were retrospectively selected under IRB and treated to 40 Gy in 5 fractions. FLASH plans were generated using a research TPS designed for dose and dose-rate optimization on a proton conformal FLASH delivery platform. A planning agent directly controlled the TPS for autonomous operation. At each optimization step, the agent evaluated dose-volume and dose-rate-volume histograms (DVHs, DRVHs) and adjusted key planning parameters. TPS then performed inverse optimization and returned updated DVHs and DRVHs. Training used a policy-based DRL algorithm guided by a reward function that enforced clinical dose goals while promoting high dose-rates (=40 GyRBE/s). The agent learned by iteratively adjusting planning parameters and updating its policy based on reward computed from changes in dose and dose-rate metrics. For evaluation, DRL-optimized plans were compared with manual FLASH plans created by an experienced physicist using the same beam geometry and planning technique. To characterize the planning policy learned via DRL, parameter adjustment trajectories and controlled perturbation analyses were performed to identify variables associated with dose and dose-rate improvement.

Results:

The DRL agent generated proton conformal FLASH plans that achieved comparable clinical dose quality while substantially improving high dose-rate coverage compared with manual plans. The mean increase in OAR volume receiving =40, 60, 80, and 100 GyRBE/s was 12.40%, 9.40%, 8.47%, and 5.10% (Table 1). The largest improvements were observed in the parotids, submandibular glands, and larynx. Feature analyses identified spot spacing, energy layer spacing and air gap as key parameters associated with dose-rate improvements.

Conclusion:

We developed and validated the first DRL-based planning agent for autonomous optimization of dose and UHDR in deliverable proton conformal FLASH. Beyond improving dose-rate performance over manual planning, the DRL-learned policy provides actionable insights to guide human FLASH planning.

Table 1. Averaged dose-rate improvements for OARs in DRL-optimized plans relative to manual FLASH plans.

	V40GyRBE/s (%)	V60GyRBE/s (%)	V80GyRBE/s (%)	V100GyRBE/s (%)
Parotids	19.32	8.00	NA	NA
Bone Mandible	6.08	8.63	4.33	NA
Oral Cavity	3.69	5.27	7.00	1.00
Constrictor	8.22	11.14	11.25	2.00
Larynx	10.21	5.67	18.00	NA
Submandibular	17.24	12.18	6.24	7.50
All OARs (Mean)	12.40	9.40	8.47	5.10