The representer theorem for Hilbert spaces: a necessary and sufficient condition

TL;DR

This paper characterizes when regularization functionals in Hilbert spaces admit a linear representer theorem, extending previous results by relaxing differentiability assumptions and providing a dimension-independent proof.

Contribution

It provides a necessary and sufficient condition for the representer theorem in Hilbert spaces under weaker assumptions than differentiability.

Findings

Regularization functionals admit a linear representer theorem if and only if they are non-decreasing functions of the norm.

The result holds under lower semi-continuity, not just differentiability.

The proof is independent of the space's dimensionality.

Abstract

A family of regularization functionals is said to admit a linear representer theorem if every member of the family admits minimizers that lie in a fixed finite dimensional subspace. A recent characterization states that a general class of regularization functionals with differentiable regularizer admits a linear representer theorem if and only if the regularization term is a non-decreasing function of the norm. In this report, we improve over such result by replacing the differentiability assumption with lower semi-continuity and deriving a proof that is independent of the dimensionality of the space.