Empirical risk minimization in inverse problems

TL;DR

This paper investigates the estimation of multivariate functions from indirect observations via linear operators, providing bounds on estimator accuracy and demonstrating optimal rates for specific inverse problems like convolution and Radon transform.

Contribution

It introduces empirical risk minimization techniques for inverse problems, deriving both upper and lower bounds, and presents a new oracle inequality for additive models.

Findings

Derived upper bounds for mean squared error of estimators.

Established optimal convergence rates for convolution and Radon transform examples.

Proposed a new oracle inequality for inverse problems in additive models.

Abstract

We study estimation of a multivariate function $f:\mathbf{R}^d\to\mathbf{R}$ when the observations are available from the function $Af$, where $A$ is a known linear operator. Both the Gaussian white noise model and density estimation are studied. We define an $L_2$-empirical risk functional which is used to define a $\delta$-net minimizer and a dense empirical risk minimizer. Upper bounds for the mean integrated squared error of the estimators are given. The upper bounds show how the difficulty of the estimation depends on the operator through the norm of the adjoint of the inverse of the operator and on the underlying function class through the entropy of the class. Corresponding lower bounds are also derived. As examples, we consider convolution operators and the Radon transform. In these examples, the estimators achieve the optimal rates of convergence. Furthermore, a new type of oracle inequality is given for inverse problems in additive models.