Solution of linear ill-posed problems by model selection and aggregation

TL;DR

This paper introduces two novel estimators for solving linear ill-posed inverse problems using model selection and aggregation, with theoretical guarantees and numerical validation.

Contribution

It proposes model selection and aggregation methods tailored for sparse representations in linear inverse problems, with sharp oracle inequalities and practical implementation.

Findings

Both estimators satisfy oracle inequalities with high probability and in expectation.

The Q-aggregate estimator achieves sharp oracle inequalities.

Numerical simulations demonstrate the effectiveness of the proposed methods.

Abstract

We consider a general statistical linear inverse problem, where the solution is represented via a known (possibly overcomplete) dictionary that allows its sparse representation. We propose two different approaches. A model selection estimator selects a single model by minimizing the penalized empirical risk over all possible models. By contrast with direct problems, the penalty depends on the model itself rather than on its size only as for complexity penalties. A Q-aggregate estimator averages over the entire collection of estimators with properly chosen weights. Under mild conditions on the dictionary, we establish oracle inequalities both with high probability and in expectation for the two estimators. Moreover, for the latter estimator these inequalities are sharp. The proposed procedures are implemented numerically and their performance is assessed by a simulation study.