Main Session
Sep
28
PQA 03 - Digital Health Innovation and Informatics, Patient Safety & Quality, and Radiation and Cancer Biology
2634 - Size-Stratified External Validation of Three FDA-Cleared AI Algorithms for Brain Metastasis Detection and Segmentation
Presenter(s)
Naseem Ud Din, PhD - Miami Cancer Institute: Baptist Health South Florida, Miami, FL
N. Ud Din1, E. Y. Y. Akdemir2, R. Herrera1, M. D. Hall1,3, D. J. Wieczorek1,3, Y. Lee1,3, R. P. Tolakanahalli1,3, A. Gutierrez1,3, E. Bander4,5, M. W. McDermott4,5, M. P. Mehta1,3, and R. Kotecha1,3; 1Department of Radiation Oncology, Miami Cancer Institute, Baptist Health South Florida, Miami, FL, 2Miami Cancer Institute, Miami, FL, 3Department of Oncological Sciences, Herbert Wertheim College of Medicine, Florida International University, Miami, FL, 4Department of Neurosurgery, Miami Neuroscience Institute, Baptist Health South Florida, Miami, FL, 5Department of Neuroscience, Herbert Wertheim College of Medicine, Florida International University, Miami, FL
Purpose/Objective(s):
With the expanding role of artificial intelligence (AI) in radiation oncology, automated brain metastasis detection and segmentation for stereotactic radiosurgery (SRS) planning has gained increasing clinical relevance. We evaluated three AI platforms, cleared by the FDA in 2025, for detection and segmentation of brain metastasis to provide comparative estimates among the platforms.Materials/Methods:
High-resolution, T1-weighted MPRAGE images were analyzed using three FDA-cleared AI platforms. Lesions were classified as true positive (TP) or false positive (FP) relative to ground truth (GT) contours derived from clinical treatment plans following multidisciplinary review (neuroradiology, radiation oncology, neurosurgery). Lesion-level sensitivity and positive predictive value (PPV) were calculated. Segmentation accuracy was assessed using the volumetric Dice similarity coefficient (DSC). Metrics are reported as mean ± standard deviation (SD) and median [interquartile range (IQR)].Results:
The external validation dataset included 50 patients with 236 brain metastases (median diameter 0.46 cm [IQR:0.29-0.83]; median volume 0.05 cc [IQR:0.01-0.30]). Mean patient-level sensitivity differed significantly among platforms (A: 81.4 ± 27.7 %, B: 74.4 ± 31.6 %, C: 69.2 ± 33.3 % p=0.0003). Lesion-level sensitivity was highest for Platform A (65.3%), followed by B (54.7%) and C (52.1%). Lesion-level PPV was similarly high for A (93.3%) and B (92.1%) but lower for C (77.0%) (p<0.001). Platform B achieved the highest overall segmentation accuracy (mean DSC 0.81 ± 0.13; median 0.84 [IQR: 0.78-0.89]), outperforming A (0.69 ± 0.21; 0.76 [0.59-0.87]) and C (0.59 ± 0.29; 0.72 [0.33-0.84]). When stratified by lesion size (0.5 cm diameter threshold), all platforms showed significantly improved performance for lesions > 0.5 cm (p<0.001). Performance based on size magnitude varied substantially: Platform C exhibited the greatest performance gap with median DSC increasing from 0.35 [0.18-0.67] for lesions < 0.5 cm to 0.83 [0.73-0.89]) for lesions > 0.5 cm. Platform B exhibited comparatively smaller variation across size groups (0.78 [0.68-0.82] vs 0.87 [0.83-0.89]), while platform A demonstrated intermediate size sensitivity (0.60 [0.44-0.74] vs 0.86 [0.79-0.89]).Conclusion:
Across all three FDA-cleared platforms, segmentation accuracy varied substantially based on lesion sizes. Performance degradation in smaller metastases differed between systems. These findings underscore the importance of external validation and size-stratified assessment prior to integrating automated brain metastasis segmentation tools into SRS workflows.