Regularized Learning Schemes in Feature Banach Spaces

TL;DR

This paper introduces a unified regularized learning framework in reflexive Banach spaces, establishing new theoretical results on representer theorems and consistency, with extensions to Hilbert spaces and insights into key functional analysis tools.

Contribution

It develops a general theory for regularized empirical risk minimization in Banach spaces, including a new representer theorem and consistency results under broad conditions.

Findings

Established a new general form of the representer theorem.

Proved the consistency of regularized learning schemes in Banach spaces.

Extended results to Hilbert spaces, providing new insights.

Abstract

This paper proposes a unified framework for the investigation of constrained learning theory in reflexive Banach spaces of features via regularized empirical risk minimization. The focus is placed on Tikhonov-like regularization with totally convex functions. This broad class of regularizers provides a flexible model for various priors on the features, including in particular hard constraints and powers of Banach norms. In such context, the main results establish a new general form of the representer theorem and the consistency of the corresponding learning schemes under general conditions on the loss function, the geometry of the feature space, and the modulus of total convexity of the regularizer. In addition, the proposed analysis gives new insight into basic tools such as reproducing Banach spaces, feature maps, and universality. Even when specialized to Hilbert spaces, this framework yields new results that extend the state of the art.