Derivative-free stochastic bilevel optimization for inverse problems

TL;DR

This paper introduces a derivative-free stochastic bilevel optimization method for inverse problems, enabling learning of model parameters from data without relying on gradient information, with proven efficiency and good generalization.

Contribution

It develops a novel derivative-free approach for stochastic bilevel optimization in inverse problems, handling non-smooth, non-convex problems with theoretical complexity analysis.

Findings

Effective in signal denoising tasks

Demonstrates good generalization to unseen data

Shows computational efficiency in experiments

Abstract

Inverse problems are key issues in several scientific areas, including signal processing and medical imaging. Data-driven approaches for inverse problems aim for learning model and regularization parameters from observed data samples, and investigate their generalization properties when confronted with unseen data. This approach dictates a statistical approach to inverse problems, calling for stochastic optimization methods. In order to learn model and regularisation parameters simultaneously, we develop in this paper a stochastic bilevel optimization approach in which the lower level problem represents a variational reconstruction method formulated as a convex non-smooth optimization problem, depending on the observed sample. The upper level problem represents the learning task of the regularisation parameters. Combining the lower level and the upper level problem leads to a stochastic non-smooth and non-convex optimization problem, for which standard gradient-based methods are not straightforward to implement. Instead, we develop a unified and flexible methodology, building on a derivative-free approach, which allows us to solve the bilevel optimization problem only with samples of the objective function values. We perform a complete complexity analysis of this scheme. Numerical results on signal denoising and experimental design demonstrate the computational efficiency and the generalization properties of our method.