3706 - Development of an In-House Application that Uses Generative AI to Summarize Oncology Histories
Presenter(s)
T. Pflederer, Z. Zhang, E. Covington, L. Higgins, J. D. Nieto, C. Mayo, and J. R. Evans Jr; Department of Radiation Oncology, University of Michigan, Ann Arbor, MI
Purpose/Objective(s): While the development of the electronic health record (EHR) has led to improvements in patient care and quality, EHR systems have also led to increased physician time burden for documentation. Oncology histories are complex - generating accurate, extensive histories by gathering data manually scattered throughout the EHR contributes significantly to physician workload prior to meeting the patient in consultation. Generative artificial intelligence (GenAI) has been implemented to improve efficiency, with examples including but not limited to scribing of clinical visits. We aim to utilize GenAI to automate the generation of accurate, complex oncology histories prior to initial consultation.
Materials/Methods: We developed an in-house application that pulls pertinent elements of an oncology history (imaging, pathology, clinical notes from specific subspecialties, etc.) from the enterprise EHR reporting database in health care software, and summarizes it in a succinct format via an institutionally sponsored OpenAI-compatible application programming interface. We used a prompt engineering approach to distill the information into a condensed and readable format. The histories are organized into a chronological format. Prompts were iteratively developed and tailored for each oncologic disease site. For this study we initially focused on a small retrospective cohort of early-stage lung cancer and localized prostate cancer patients.
Results: We crafted 11 prompts for each component of an early-stage lung cancer oncologic history and 5 prompts for localized prostate cancer. We found that prompts asking for summarization, rather than interpretation, subjectively led to fewer errors. We found that hallucinations were possible – for example, if a radiology report does not comment on lymph nodes, this could be misinterpreted by AI as there being no evidence of lymphadenopathy. Errors like this were avoided if explicit instructions were included to only comment on what is stated in the radiology report. Multiple iterations of these prompts will be required and recorded as our retrospective patient population grows.
Conclusion: We developed a proof-of-concept GenAI custom software utility for summarizing oncology histories. We plan to continue prompt engineering to expand our retrospective cohort to those with more complex oncologic histories. Our preliminary results highlight the importance of prompt engineering to clinically optimize results from current generations of GenAI. We plan on testing, scoring, and modifying this software with real-time physician input for a larger prospective patient cohort in our department. As this is developed further, our goal is to significantly reduce the pre-consultation EHR burden on physicians.