168 - Advancing 3D MRI Foundation Models for Enhanced MR Image Analysis

Purpose/Objective(s): As magnetic resonance imaging (MRI) is increasingly integrated into radiotherapy, artificial intelligence (AI) models are essential for enhancing MR image analysis toward precise treatment. However, existing approaches often require training multiple AI models tailored to diverse anatomical sites, tasks, or MRI modalities, thereby complicating clinical integration. To overcome these limitations, we propose pretraining a comprehensive foundation model on an expansive dataset encompassing diverse types of MR data. This approach enables the model to learn general representations that can be fine-tuned to significantly boost performance across a broad spectrum of MR image analysis applications.

Materials/Methods: Our approach involves developing a 3D MRI foundation model that employs an autoencoder architecture to learn robust representations from a dataset of over 131,170 3D MRI volumes. To ensure the relevance of the learned features across different imaging scenarios, the model leverages anatomical site-independent imaging descriptions, thereby constraining the semantic distribution across visual modalities. This methodology has resulted in the creation of the largest known 3D MRI pre-training dataset to date. We evaluated the utility and flexibility of our model across 3 MR image analysis tasks: cancer classification, tumor and organ segmentation, and image registration. These evaluations were conducted using a diverse set of 14 downstream MRI datasets (e.g., T1, T2, FLAIR) with a total of 6,895 patients across the brain, breast, heart, pancreas, liver and prostate to ensure comprehensive testing.

Results: Initial testing of our 3D MRI foundation model across multiple benchmarking tasks has demonstrated substantial flexibility and potential to enhance MRI analysis. Specifically, the model achieved a mean increase of 4.8% in classification accuracy across 3 datasets with 2,912 patients, an average improvement of 1.0% in segmentation accuracy across 9 datasets with 2,994 patients, and an average enhancement of 4.0% in registration accuracy across 2 datasets with 989 patients, all compared to baseline models built from scratch. Moreover, our model consistently outperforms other publicly available research models in MR image classification, segmentation, and registration tasks.

Conclusion: This study underscores the transformative potential of large-scale pre-training techniques for advancing 3D MR image analysis. The 3D MRI foundation model we developed not only offers a groundbreaking approach to MR image analysis but also captures the intricate complexity of 3D MRI data, making it a highly valuable resource for researchers, developers, and clinicians alike. By making our model weights, code, and datasets openly available, we aim to enhance the adaptability and reliability of AI-driven solutions in 3D MRI. This initiative is set to pave the way for more precise and effective patient care in clinical environments.