1088 - PURE-MRI: An International Study Assessing Physician Accuracy in Delineating the Prostate and Urethra on Prostate MRI
Presenter(s)
L. Nguyen1,2, Y. Song1,3, A. Dornisch1, M. Baxter1, T. Barrett4, A. M. Dale5, M. Harisinghani6, S. C. Kamran7, M. A. Liss8, R. T. Dess9, D. Margolis10, E. P. Weinberg11, and T. M. Seibert1,12; 1Department of Radiation Medicine and Applied Sciences, University of California San Diego, La Jolla, CA, 2University of California San Diego School of Medicine, La Jolla, CA, 3Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA, 4Department of Radiology, University of Cambridge, Cambridge, United Kingdom, 5Department of Radiology, University of California San Diego, La Jolla, CA, 6Department of Radiology, Massachusetts General Hospital, Boston, MA, 7Department of Radiation Oncology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, 8Department of Urology, University of California San Diego, La Jolla, CA, 9Department of Radiation Oncology, University of Michigan, Ann Arbor, MI, 10Department of Radiology, Cornell University, Ithaca, NY, USA, Ithaca, NY, 11Department of Clinical Imaging Sciences, University of Rochester Medical Center, Rochester, NY, 12Department of Bioengineering, University of California San Diego, La Jolla, CA
Purpose/Objective(s):
Precise delineation of genitourinary (GU) structures during prostate cancer (PCa) care is critical to optimize treatment delivery while minimizing toxicity and injury. The Prostate and UREthra on MRI (PURE-MRI) study is an international, prospective study to assess physicians’ accuracy segmenting the prostate and urethra on MRI.Materials/Methods: Physicians who diagnose or treat PCa were invited to contour the prostate and urethra on up to 4 patient cases using standard T2-weighted MRI (all planes). These contours were compared to reference consensus segmentations produced by a multidisciplinary panel of experts. Performance of a validated prostate auto-segmentation AI tool was also evaluated. We assessed contour accuracy with Dice similarity coefficient (DSC), deviation from boundary (mm), absolute volume difference (%), and overlap (%). A mixed effects model was used to evaluate potential associations between contour performance and physician specialty, GU focus, or clinical experience.
Results: 59 radiation oncologists, radiologists, and urologists (from 11 countries) created a total of 212 structure segmentations (108 prostate, 104 urethra). DSC for the prostate, reported as median (min, max), was 0.92 (0.90, 0.94) for physicians, with no clear effect of clinical experience or focus. Maximum deviation inside (cutting into expert contour), maximum deviation outside (extending beyond expert contour), and mean deviation (per case) from the reference prostate were 3.4 mm (1.0, 12.4), 5.3 mm (2.4, 7.0), and 1.6 mm (1.3, 17.3), respectively. By comparison, the prostate auto-segmentation tool had DSC, max deviation inside, max distance outside, and mean deviation per case of 0.95 (0.94, 0.96), 3 mm, 3.9 mm (3.1, 4.9), and 1.2 mm (1.1, 1.5), respectively. Physician performance was considerably worse for the urethra, with DSC of 0.33 (0.03, 0.69). No AI tool was tested for the urethra.
Conclusion: Physicians contour the prostate on MRI with overall DSC >0.9, though contours typically had at least one error >5 mm and sometimes >10 mm (Table 1). These patterns were observed regardless of clinical experience, specialty, or clinical focus. A current AI tool performs well enough for clinical use, given comparable accuracy to practicing physicians. In contrast, urethra segmentation on MRI is challenging. More training, better imaging, and/or AI tools may be necessary to achieve accurate and consistent results for the urethra.
Abstract 1088 - Table 1: Physician and AI contour accuracy metrics, reported as median (min, max)| Accuracy Metric | AI Prostate Contours (n=4) | Physician Prostate Contours (n=108) | Physician Urethra Contours (n=104) | |
| DSC | 0.95 (0.94, 0.96) | 0.92 (0.62, 0.95) | 0.33 (0.03, 0.69) | |
| Error Deviation (mm) | Max Inside | 3.0 | 3.4 (1.0, 12.4) | |
| Max Outside | 3.9 (3.1, 4.9) | 5.3 (2.4, 7.0) | ||
| Mean | 1.2 (1.1, 1.5) | 1.6 (1.3, 17.3) | ||
| Volume Difference (%) | 4 (3, 7) | 7 (0, 48) | ||
| Overlap (%) | 35 (3, 96) | |||