Joint learning of variational representations and solvers for inverse problems with partially-observed data

TL;DR

This paper introduces an end-to-end neural framework that jointly learns variational regularization models and solvers for inverse problems, improving reconstruction quality on incomplete data.

Contribution

It proposes a novel supervised approach to learn both the variational cost and the solver as neural networks, enabling data-driven discovery of variational models for inverse problems.

Findings

Significant improvement in image inpainting and time series interpolation.

Outperforms direct minimization of known variational models.

Demonstrates the effectiveness of joint learning in inverse problems.

Abstract

Designing appropriate variational regularization schemes is a crucial part of solving inverse problems, making them better-posed and guaranteeing that the solution of the associated optimization problem satisfies desirable properties. Recently, learning-based strategies have appeared to be very efficient for solving inverse problems, by learning direct inversion schemes or plug-and-play regularizers from available pairs of true states and observations. In this paper, we go a step further and design an end-to-end framework allowing to learn actual variational frameworks for inverse problems in such a supervised setting. The variational cost and the gradient-based solver are both stated as neural networks using automatic differentiation for the latter. We can jointly learn both components to minimize the data reconstruction error on the true states. This leads to a data-driven discovery of variational models. We consider an application to inverse problems with incomplete datasets (image inpainting and multivariate time series interpolation). We experimentally illustrate that this framework can lead to a significant gain in terms of reconstruction performance, including w.r.t. the direct minimization of the variational formulation derived from the known generative model.