Nexus-INR: Diverse Knowledge-guided Arbitrary-Scale Multimodal Medical Image Super-Resolution

TL;DR

Nexus-INR introduces a knowledge-guided, multi-component framework for arbitrary-scale medical image super-resolution that effectively leverages multi-modal data and anatomical semantics to improve image quality and segmentation accuracy.

Contribution

It presents a novel multi-component architecture with knowledge distillation and semantic embedding for flexible, high-quality medical image super-resolution.

Findings

Outperforms state-of-the-art methods on BraTS2020 dataset

Enhances downstream segmentation performance

Achieves high-quality arbitrary-scale super-resolution

Abstract

Arbitrary-resolution super-resolution (ARSR) provides crucial flexibility for medical image analysis by adapting to diverse spatial resolutions. However, traditional CNN-based methods are inherently ill-suited for ARSR, as they are typically designed for fixed upsampling factors. While INR-based methods overcome this limitation, they still struggle to effectively process and leverage multi-modal images with varying resolutions and details. In this paper, we propose Nexus-INR, a Diverse Knowledge-guided ARSR framework, which employs varied information and downstream tasks to achieve high-quality, adaptive-resolution medical image super-resolution. Specifically, Nexus-INR contains three key components. A dual-branch encoder with an auxiliary classification task to effectively disentangle shared anatomical structures and modality-specific features; a knowledge distillation module using cross-modal attention that guides low-resolution modality reconstruction with high-resolution reference, enhanced by self-supervised consistency loss; an integrated segmentation module that embeds anatomical semantics to improve both reconstruction quality and downstream segmentation performance. Experiments on the BraTS2020 dataset for both super-resolution and downstream segmentation demonstrate that Nexus-INR outperforms state-of-the-art methods across various metrics.