2612 - Prostate Cancer Clinical Quality Assessment across the Veterans Health Administration: Results from the VHA Radiation Oncology Quality Surveillance Program (VHA-ROQS)

Purpose/Objective(s): The VHA-ROQS program, developed in partnership with the American Society for Radiation Oncology (ASTRO), evaluates adherence to evidence-based clinical quality measures (CQMs) across VA radiation oncology practices. We report national performance on prostate cancer CQMs and identify documentation and care gaps to inform quality improvement initiatives.

Materials/Methods: Case report forms were distributed to 41 VA radiation oncology sites, requesting data for five patients per radiation oncologist (minimum 10 per site). Thirty-nine sites submitted data for 515 prostate cancer patients treated with moderate hypofractionation (60–70 Gy in 20–30 fractions). . Clinical notes (from initial consult, simulation, treatment planning, OTV, EOT and follow-up) and DICOM/DICOM-RT data files were collected and anonymized. Abstraction of structured data from clinical notes was done by an AI-enabled platform., with manual review and adjudication to address extraction inaccuracies and site-specific documentation variability. CQMs, developed by ASTRO disease site-specific expert panels, were scored using predefined binary decision trees.

Results:

Overall adherence varied widely across measures. High performance was observed for several treatment planning and dose-related metrics (92–99%). Documentation of initial evaluation to include NCCN risk, PSA, Gleason score and nodal stage scored 93%. Pre-treatment quality of life (QoL) documentation was achieved in 81% of patients, though instruments were inconsistently documented in follow-up (59%). MRI documentation prior to treatment was present in 82%, and PET documentation for high-risk disease in 62%. Multidisciplinary evaluation or urology involvement was documented in 86%.

Lower adherence was observed in process-based measures, including documentation of comprehensive discussion of all treatment options (33%), active surveillance counseling in low-risk patients (79%), and nodal treatment documentation (15%). Several measures were excluded due to insufficient documentation (e.g., genomic classifiers, bone health assessment).

Key challenges included non-standardized EMR note titles, variable abbreviations, inconsistent TG-263 structure nomenclature, and limited capture of required elements. While AI-assisted abstraction improved efficiency, manual review remained necessary to fix AI based hallucinations and abnormal findings.

Conclusion: Prostate CQM performance across VA sites demonstrates strong adherence to dose and planning metrics but reveals variability in clinical documentation. Standardized note templates and structured data capture, such as through VA-HINGE-based workflows, may enhance measure compliance, reduce abstraction burden, and improve longitudinal quality surveillance. Furthermore, our study provides insights into the challenges and strategies for enterprise-wide quality surveillance in other settings.