On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstruction

TL;DR

This paper explores using vision-language foundation models to improve undersampled MRI reconstruction by incorporating high-level semantic information, leading to better anatomical detail preservation and perceptual quality.

Contribution

It introduces a semantic distribution-guided reconstruction framework that leverages pre-trained vision-language models and contrastive learning to enhance MRI image quality with semantic priors.

Findings

Semantic priors improve anatomical detail preservation.

Image-language priors enable high-level control over reconstructions.

Contrastive objective aligns features with semantic distributions effectively.

Abstract

Purpose: To investigate whether a vision-language foundation model can enhance undersampled MRI reconstruction by providing high-level contextual information beyond conventional priors. Methods: We proposed a semantic distribution-guided reconstruction framework that uses a pre-trained vision-language foundation model to encode both the reconstructed image and auxiliary information into high-level semantic features. A contrastive objective aligns the reconstructed representation with the target semantic distribution, ensuring consistency with high-level perceptual cues. The proposed objective works with various deep learning-based reconstruction methods and can flexibly incorporate semantic priors from multimodal sources. To test the effectiveness of these semantic priors, we evaluated reconstruction results guided by priors derived from either image-only or image-language auxiliary information. Results: Experiments on knee and brain datasets demonstrate that semantic priors from images preserve fine anatomical structures and achieve superior perceptual quality, as reflected in lower LPIPS values, higher Tenengrad scores, and improved scores in the reader study, compared with conventional regularization. The image-language information further expands the semantic distribution and enables high-level control over reconstruction attributes. Across all evaluations, the contrastive objective consistently guided the reconstructed features toward the desired semantic distributions while maintaining data fidelity, demonstrating the effectiveness of the proposed optimization framework. Conclusion: The study highlights that vision-language foundation models can improve undersampled MRI reconstruction through semantic-space optimization.