Functional learning through kernels

TL;DR

This paper explores the theoretical foundations of functional learning, generalizing reproducing kernel Hilbert spaces to non-Hilbertian sets and establishing a framework for kernel design in learning machines.

Contribution

It introduces a generalized theory of reproducing kernel spaces beyond Hilbert spaces and derives a representer theorem applicable to these sets, enabling kernel-based learning.

Findings

Generalized reproducing kernel sets are necessary for learning machines.

A new representer theorem applies to non-Hilbertian sets.

Examples of kernels and reproducing sets are provided.

Abstract

This paper reviews the functional aspects of statistical learning theory. The main point under consideration is the nature of the hypothesis set when no prior information is available but data. Within this framework we first discuss about the hypothesis set: it is a vectorial space, it is a set of pointwise defined functions, and the evaluation functional on this set is a continuous mapping. Based on these principles an original theory is developed generalizing the notion of reproduction kernel Hilbert space to non hilbertian sets. Then it is shown that the hypothesis set of any learning machine has to be a generalized reproducing set. Therefore, thanks to a general "representer theorem", the solution of the learning problem is still a linear combination of a kernel. Furthermore, a way to design these kernels is given. To illustrate this framework some examples of such reproducing sets and kernels are given.