Solving a Machine Learning Regression Problem Based on the Theory of Random Functions

TL;DR

This paper develops a theoretically grounded regression method based on the symmetry properties of random functions, deriving a kernel that generalizes polyharmonic splines without empirical selection.

Contribution

It introduces a regression approach derived from indifference principles, establishing a theoretical foundation for smoothing and interpolation methods based on symmetry invariances.

Findings

Kernel coincides with a generalized polyharmonic spline

Method derived analytically from symmetry postulates

Provides a theoretical basis for optimal smoothing methods

Abstract

This paper studies a machine learning regression problem as a multivariate approximation problem using the framework of the theory of random functions. An ab initio derivation of a regression method is proposed, starting from postulates of indifference. It is shown that if a probability measure on an infinite-dimensional function space possesses natural symmetries (invariance under translation, rotation, scaling, and Gaussianity), then the entire solution scheme, including the kernel form, the type of regularization, and the noise parameterization, follows analytically from these postulates. The resulting kernel coincides with a generalized polyharmonic spline; however, unlike existing approaches, it is not chosen empirically but arises as a consequence of the indifference principle. This result provides a theoretical foundation for a broad class of smoothing and interpolation methods, demonstrating their optimality in the absence of a priori information.