Variational Regularization in Inverse Problems and Machine Learning

TL;DR

This paper reviews variational regularization methods for inverse problems and machine learning, highlighting theoretical properties, recent developments, and new connections between error estimates and generalization.

Contribution

It introduces novel links between inverse problem regularization and machine learning, including reinterpretations and error estimate connections.

Findings

Established connections between Bregman distance error estimates and generalization errors.

Reinterpreted machine learning problems within the regularization framework.

Provided convergence properties and quantitative estimates for variational regularization.

Abstract

This paper discusses basic results and recent developments on variational regularization methods, as developed for inverse problems. In a typical setup we review basic properties needed to obtain a convergent regularization scheme and further discuss the derivation of quantitative estimates respectively needed ingredients such as Bregman distances for convex functionals. In addition to the approach developed for inverse problems we will also discuss variational regularization in machine learning and work out some connections to the classical regularization theory. In particular we will discuss a reinterpretation of machine learning problems in the framework of regularization theory and a reinterpretation of variational methods for inverse problems in the framework of risk minimization. Moreover, we establish some previously unknown connections between error estimates in Bregman distances and generalization errors.