Empirical risk minimization in inverse problems: Extended technical version

TL;DR

This paper investigates empirical risk minimization techniques for inverse problems, providing theoretical bounds, optimal rates, and new inequalities for estimating functions from linear operator observations.

Contribution

It introduces a comprehensive analysis of empirical risk minimizers in inverse problems, including upper and lower bounds, and extends results to additive models with new oracle inequalities.

Findings

Derived upper bounds for mean squared error of estimators

Established lower bounds matching the upper bounds for certain operators

Achieved optimal convergence rates for convolution and Radon transform examples

Abstract

We study estimation of a multivariate function $f:{\bf R}^d \to {\bf R}$ when the observations are available from function $Af$, where $A$ is a known linear operator. Both the Gaussian white noise model and density estimation are studied. We define an $L_2$ empirical risk functional, which is used to define an $\delta$-net minimizer and a dense empirical risk minimizer. Upper bounds for the mean integrated squared error of the estimators are given. The upper bounds show how the difficulty of the estimation depends on the operator through the norm of the adjoint of the inverse of the operator, and on the underlying function class through the entropy of the class. Corresponding lower bounds are also derived. As examples we consider convolution operators and the Radon transform. In these examples the estimators achieve the optimal rates of convergence. Furthermore, a new type of oracle inequality is given for inverse problems in additive models.