Task Transformer Network for Joint MRI Reconstruction and Super-Resolution

TL;DR

This paper introduces T²Net, an end-to-end transformer-based model that jointly performs MRI reconstruction and super-resolution, leveraging shared features to produce higher-quality, artifact-free images from undersampled data.

Contribution

The novel joint task transformer network effectively combines MRI reconstruction and super-resolution, outperforming sequential methods by sharing representations and transferring task-specific information.

Findings

Significantly outperforms sequential methods in quality metrics

Achieves higher resolution and fewer artifacts in MRI images

Demonstrates effective joint feature learning between tasks

Abstract

The core problem of Magnetic Resonance Imaging (MRI) is the trade off between acceleration and image quality. Image reconstruction and super-resolution are two crucial techniques in Magnetic Resonance Imaging (MRI). Current methods are designed to perform these tasks separately, ignoring the correlations between them. In this work, we propose an end-to-end task transformer network (T$^2$Net) for joint MRI reconstruction and super-resolution, which allows representations and feature transmission to be shared between multiple task to achieve higher-quality, super-resolved and motion-artifacts-free images from highly undersampled and degenerated MRI data. Our framework combines both reconstruction and super-resolution, divided into two sub-branches, whose features are expressed as queries and keys. Specifically, we encourage joint feature learning between the two tasks, thereby transferring accurate task information. We first use two separate CNN branches to extract task-specific features. Then, a task transformer module is designed to embed and synthesize the relevance between the two tasks. Experimental results show that our multi-task model significantly outperforms advanced sequential methods, both quantitatively and qualitatively.