Synthesizing dynamic MRI using long-term recurrent convolutional networks

TL;DR

This paper introduces a neural network-based method to synthesize high frame rate dynamic MRI from ultrasound signals, enabling real-time imaging of respiratory organ motion without additional MRI input.

Contribution

It presents a novel long-term recurrent convolutional neural network that converts ultrasound signals into high frame rate MRI, trained with paired ultrasound and MRI data.

Findings

Achieved 100 frames per second MRI synthesis from ultrasound data.

Validated on 7 subjects demonstrating effective motion prediction.

No additional MRI input needed during inference.

Abstract

A method is proposed for converting raw ultrasound signals of respiratory organ motion into high frame rate dynamic MRI using a long-term recurrent convolutional neural network. Ultrasound signals were acquired using a single-element transducer, referred to here as `organ-configuration motion' (OCM) sensor, while sagittal MR images were simultaneously acquired. Both streams of data were used for training a cascade of convolutional layers, to extract relevant features from raw ultrasound, followed by a recurrent neural network, to learn its temporal dynamics. The network was trained with MR images on the output, and was employed to predict MR images at a temporal resolution of 100 frames per second, based on ultrasound input alone, without any further MR scanner input. The method was validated on 7 subjects.