One Network to Solve Them All: A Sequential Multi-Task Joint Learning Network Framework for MR Imaging Pipeline

TL;DR

This paper introduces a sequential multi-task learning framework that jointly optimizes MRI sampling, reconstruction, and segmentation in an end-to-end differentiable pipeline, leveraging their interrelations for improved clinical outcomes.

Contribution

It proposes a novel integrated network that models the mutual influences among MRI tasks, enabling end-to-end training for enhanced performance over separate task processing.

Findings

Outperforms state-of-the-art methods in MRI reconstruction and segmentation

Effectively models task interrelations for mutual benefit

Achieves superior results on MRB dataset

Abstract

Magnetic resonance imaging (MRI) acquisition, reconstruction, and segmentation are usually processed independently in the conventional practice of MRI workflow. It is easy to notice that there are significant relevances among these tasks and this procedure artificially cuts off these potential connections, which may lead to losing clinically important information for the final diagnosis. To involve these potential relations for further performance improvement, a sequential multi-task joint learning network model is proposed to train a combined end-to-end pipeline in a differentiable way, aiming at exploring the mutual influence among those tasks simultaneously. Our design consists of three cascaded modules: 1) deep sampling pattern learning module optimizes the $k$-space sampling pattern with predetermined sampling rate; 2) deep reconstruction module is dedicated to reconstructing MR images from the undersampled data using the learned sampling pattern; 3) deep segmentation module encodes MR images reconstructed from the previous module to segment the interested tissues. The proposed model retrieves the latently interactive and cyclic relations among those tasks, from which each task will be mutually beneficial. The proposed framework is verified on MRB dataset, which achieves superior performance on other SOTA methods in terms of both reconstruction and segmentation.