Featured Reproducing Kernel Banach Spaces for Learning and Neural Networks

TL;DR

This paper develops a theoretical framework for kernel-based learning in Banach spaces, extending classical Hilbert space methods to more general settings relevant for neural networks and modern models.

Contribution

It introduces featured reproducing kernel Banach spaces, establishing conditions for feature maps and representer theorems beyond Hilbert spaces, and connects neural networks to these spaces.

Findings

Established conditions for feature maps in Banach spaces.

Proved existence and representer theorems for Banach space learning.

Linked neural networks to featured Banach spaces.

Abstract

Reproducing kernel Hilbert spaces provide a foundational framework for kernel-based learning, where regularization and interpolation problems admit finite-dimensional solutions through classical representer theorems. Many modern learning models, however -- including fixed-architecture neural networks equipped with non-quadratic norms -- naturally give rise to non-Hilbertian geometries that fall outside this setting. In Banach spaces, continuity of point-evaluation functionals alone is insufficient to guarantee feature representations or kernel-based learning formulations. In this work, we develop a functional-analytic framework for learning in Banach spaces based on the notion of featured reproducing kernel Banach spaces. We identify the precise structural conditions under which feature maps, kernel constructions, and representer-type results can be recovered beyond the Hilbertian regime. Within this framework, supervised learning is formulated as a minimal-norm interpolation or regularization problem, and existence results together with conditional representer theorems are established. We further extend the theory to vector-valued featured reproducing kernel Banach spaces and show that fixed-architecture neural networks naturally induce special instances of such spaces. This provides a unified function-space perspective on kernel methods and neural networks and clarifies when kernel-based learning principles extend beyond reproducing kernel Hilbert spaces.