The connection between Bayesian estimation of a Gaussian random field and RKHS

TL;DR

This paper explores the statistical interpretation of RKHS-based function estimation from noisy data, extending known Gaussian process links to more general loss functions like Huber and Vapnik.

Contribution

It generalizes the Bayesian interpretation of RKHS estimators beyond quadratic loss to include absolute, Vapnik, and Huber losses, linking MAP estimates to RKHS solutions.

Findings

RKHS estimates correspond to MAP estimates under various loss functions

Extension of Gaussian random field interpretation to non-quadratic losses

Theoretical foundation for using RKHS in robust and alternative loss-based estimation

Abstract

Reconstruction of a function from noisy data is often formulated as a regularized optimization problem over an infinite-dimensional reproducing kernel Hilbert space (RKHS). The solution describes the observed data and has a small RKHS norm. When the data fit is measured using a quadratic loss, this estimator has a known statistical interpretation. Given the noisy measurements, the RKHS estimate represents the posterior mean (minimum variance estimate) of a Gaussian random field with covariance proportional to the kernel associated with the RKHS. In this paper, we provide a statistical interpretation when more general losses are used, such as absolute value, Vapnik or Huber. Specifically, for any finite set of sampling locations (including where the data were collected), the MAP estimate for the signal samples is given by the RKHS estimate evaluated at these locations.