When is there a Representer Theorem? Nondifferentiable Regularisers and Banach spaces

TL;DR

This paper extends the conditions under which the representer theorem applies to nondifferentiable regularisers in Banach spaces, broadening the theoretical foundation for kernel methods beyond Hilbert spaces.

Contribution

It provides necessary and sufficient conditions for the representer theorem in uniformly convex and smooth Banach spaces with nondifferentiable regularisers, generalizing previous results.

Findings

Extended the representer theorem to Banach spaces with nondifferentiable regularisers

Showed the solution depends only on the function space, not the regulariser

Provided a more complete theoretical framework for kernel methods

Abstract

We consider a general regularised interpolation problem for learning a parameter vector from data. The well known representer theorem says that under certain conditions on the regulariser there exists a solution in the linear span of the data points. This is the core of kernel methods in machine learning as it makes the problem computationally tractable. Necessary and sufficient conditions for differentiable regularisers on Hilbert spaces to admit a representer theorem have been proved. We extend those results to nondifferentiable regularisers on uniformly convex and uniformly smooth Banach spaces. This gives a (more) complete answer to the question when there is a representer theorem. We then note that for regularised interpolation in fact the solution is determined by the function space alone and independent of the regulariser, making the extension to Banach spaces even more valuable.