On the connection between supervised learning and linear inverse problems

TL;DR

This paper explores the deep connection between supervised learning and linear inverse problems, showing they can both be viewed as approximation problems in a reproducing kernel Hilbert space, and analyzes their convergence properties.

Contribution

It establishes a formal correspondence between supervised learning and linear inverse problems within RKHS, linking their solutions and convergence behaviors.

Findings

Supervised learning and inverse problems are equivalent to approximation problems in RKHS.

Discretized solutions converge to the continuous problem solutions under both deterministic and statistical frameworks.

Convergence rates relate noise levels to sample sizes, bridging statistical and deterministic bounds.

Abstract

In this paper we investigate the connection between supervised learning and linear inverse problems. We first show that a linear inverse problem can be view as a function approximation problem in a reproducing kernel Hilbert space (RKHS) and then we prove that to each of these approximation problems corresponds a class of inverse problems. Analogously, we show that Tikhonov solutions of this class correspond to the Tikhonov solution of the approximation problem. Thanks to this correspondence, we show that supervised learning and linear discrete inverse problems can be thought of as two instances of the approximation problem in a RKHS. These instances are formalized by means of a sampling operator which takes into account both deterministic and random samples and leads to discretized problems. We then analyze the discretized problems and we study the convergence of their solutions to the ones of the approximation problem in a RKHS, both in the deterministic and statistical framework. Finally, we prove there exists a relation between the convergence rates computed with respect to the noise level and the ones computed with respect to the number of samples. This allows us to compare upper and lower bounds given in the statistical learning and in the deterministic infinite dimensional inverse problems theory.