Main Session
Sep 30
PQA 09 - Hematologic Malignancies, Health Services Research, Digital Health Innovation and Informatics

3614 - MR-Linac-Based Adaptive and Non-Adaptive SBRT for Prostate and Pancreatic Cancer vs. Conventional-Linac-Based Treatment: A Time-Driven Activity-Based Costing Analysis

04:00pm - 05:00pm PT
Hall F
Screen: 8

Presenter(s)

Zubir Rentiya, MD, MSc - Cleveland Clinic Foundation, Cleveland, OH

Y. Elnady1, B. Underwood1, L. Mancuso1, V. Lovasz1, C. Smith1, D. B. Wiant2, M. M. Matuszak3, J. R. Evans Jr3, and C. D. Jahraus4; 1Fuse Oncology, Boone, NC, 2Cone Health Cancer Center, Greensboro, NC, 3Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, 4Generations Radiotherapy and Oncology PC, Alabaster, AL

Purpose/Objective(s): The consult preparation workflow in Radiation Oncology is often time-consuming and error-prone due to the need to assess voluminous documentation, extract relevant data, and manually prepare an HPI. Many radiation oncologists spend hours each week beyond clinical duties on this repetitive task. Advancements in generative AI offer an opportunity to streamline this process, while improving accuracy and potentially decreasing physician burn-out. This step is also critical for patient treatment decision-making, as the extracted data often informs the plan of care selected. This study evaluates a commercial AI-driven HPI generation system designed to process documents (scanned documents, faxes, pdfs, docs, etc...), extract clinical data, identify discrepancies, automate HPI generation, and enhance physicians' understanding of patient history. In this study we utilize a battery of industry standardized LLM quality assessment metrics to test the accuracy of this approach.

Materials/Methods: The AI-powered workflow uses disease-specific HPI templates and optical character recognition to facilitate smart retrieval of structured data, which is then used to generate the HPI citations. To evaluate the AI-data extraction and HPI generation quality, we compare 200 AI-generated outputs with real patient records using six key metrics:

  1. Faithfulness – Measures factual consistency against original records
  2. Hallucination – Identifies potential introduction of non-existent information
  3. Bias – Detects disparities in language or representation
  4. Tone Alignment – Assesses adherence to clinical documentation style
  5. Keyword Presence – Evaluates inclusion of predefined medical terms
  6. Answer Relevancy – Measures focus on clinical context without unnecessary content

Results: Our comparison of AI-generated HPIs to the patient record revealed the following:

  1. Faithfulness: Strong factual consistency
  2. Hallucination Rate: <1%, ensuring minimal misinformation
  3. Bias: 0%, confirming neutrality
  4. Tone Alignment: 100% compliance
  5. Keyword Presence: 84% match rate, subject to source material variability
  6. Answer Relevancy: 97.3%, ensuring precise and relevant documentation
These results show that the AI-Driven workflow can support HPI documentation completion while maintaining quality and consistency.

Conclusion: These findings highlight the potential of AI-driven solutions in improving documentation efficiency while maintaining high standards of accuracy in radiation oncology. Furthermore, with these promising results in structured oncologic data extraction, we believe this approach can have broader applications in informing treatment pathway selection in the future.