MoDL: Model Based Deep Learning Architecture for Inverse Problems

TL;DR

MoDL introduces a model-based deep learning framework for inverse problems that explicitly incorporates the forward model, enabling smaller networks, faster convergence, and improved performance with less training data.

Contribution

The paper presents a novel end-to-end trainable deep learning architecture that shares weights across iterations and channels, explicitly models the forward process, and integrates numerical optimization for data consistency.

Findings

Fewer training data needed due to explicit forward model incorporation.

Faster convergence achieved by using conjugate gradient within the network.

Reduced memory footprint and overfitting risk with shared weights.

Abstract

We introduce a model-based image reconstruction framework with a convolution neural network (CNN) based regularization prior. The proposed formulation provides a systematic approach for deriving deep architectures for inverse problems with the arbitrary structure. Since the forward model is explicitly accounted for, a smaller network with fewer parameters is sufficient to capture the image information compared to black-box deep learning approaches, thus reducing the demand for training data and training time. Since we rely on end-to-end training, the CNN weights are customized to the forward model, thus offering improved performance over approaches that rely on pre-trained denoisers. The main difference of the framework from existing end-to-end training strategies is the sharing of the network weights across iterations and channels. Our experiments show that the decoupling of the number of iterations from the network complexity offered by this approach provides benefits including lower demand for training data, reduced risk of overfitting, and implementations with significantly reduced memory footprint. We propose to enforce data-consistency by using numerical optimization blocks such as conjugate gradients algorithm within the network; this approach offers faster convergence per iteration, compared to methods that rely on proximal gradients steps to enforce data consistency. Our experiments show that the faster convergence translates to improved performance, especially when the available GPU memory restricts the number of iterations.