341 - Re-Engaging with Peer Review: Impact of a Restructured Contour/Plan Review Format in Radiation Oncology

Purpose/Objective(s): Peer review of physician-approved contours and plans is essential to quality assurance in radiation oncology. While daily contour/plan review has been central to our institution's quality program, we observed opportunities to deepen engagement among participants. We transitioned from our standard format to an enhanced review mandating 1) assigned and monitored attendance, 2) verified contour/plan approval from an in-session physician, 3) a designated rounds lead, 4) video-on for remote participants, 5) standardized dosimetrist presentations. We sought to compare clinically meaningful feedback between formats.

Materials/Methods: We analyzed prospectively graded peer review data across 8 campuses over 35 months spanning the standard era (n=7,261) and enhanced era (n=3,354), totaling 10,615 cases. Each case was graded at time of presentation: A (Pass), B (Conditional Pass; approved with modifications), or C (Fail; major revision required). Reviewer feedback was classified, blinded to era, using predefined criteria as administrative (naming conventions, boolean requests, contour approval) or clinical (target volume modifications, margin changes, dose adjustments, OAR concerns, coverage issues, technique changes, safety interventions). The primary endpoint was the proportion of cases with clinical feedback (grades B or C with clinical content). Statistical comparisons used chi-square testing with odds ratios (OR) and 95% confidence intervals (CI).

Results: Overall grade distributions shifted significantly from standard to enhanced review. Grade A decreased from 70.0% to 49.7% (p<0.0001). Grade B increased from 29.8% to 44.9% (p<0.0001). Grade C increased from 0.21% to 5.43% (p<0.0001). Combined B+C rose from 30.0% to 50.3% (p<0.0001).

Restricting analysis to clinical feedback, enhanced review demonstrated significant increases across all categories. Clinical B tripled from 12.2% to 36.4% (883/7,261 vs 1,221/3,354; OR 4.13, 95% CI 3.72–4.58, p<0.0001). Clinical C rose from 0.12% to 3.58% (9/7,261 vs 120/3,354; OR 29.8, 95% CI 15.1–58.8, p<0.0001). Pooled clinical B+C increased from 12.3% to 40.0% (892/7,261 vs 1,341/3,354; OR 4.74, 95% CI 4.28–5.25, p<0.0001). Administrative-only feedback decreased from 87.7% to 60.0%. Among Grade B cases, 81.1% contained clinical feedback in the enhanced era versus 40.9% previously. Clinical feedback rates increased across 2D, 3D, IMRT and SBRT cases independently.

Conclusion: Transitioning to a structured peer review format was associated with a nearly 30% absolute increase in cases given clinical feedback requiring contour/plan modification (OR 4.74, p<0.0001). There were no concurrent changes in operations, workflow, or case mix during this period. While these findings suggest enhanced review was associated with increased clinical feedback from reviewing peers, further investigation is ongoing to confirm the reason for this increase, rate of feedback acceptance, and whether negative effects also resulted.