Singular Value Decomposition of Operators on Reproducing Kernel Hilbert Spaces

TL;DR

This paper develops a rigorous mathematical framework for the eigenvalue and singular value decompositions of operators on reproducing kernel Hilbert spaces, which are crucial in many machine learning applications.

Contribution

It extends existing methods by providing a solid functional analytic foundation for SVD of RKHS operators, enhancing the theoretical understanding and computational techniques.

Findings

Established a functional analytic foundation for SVD of RKHS operators

Extended eigenvalue decomposition methods to singular value decomposition

Illustrated results with simple guiding examples

Abstract

Reproducing kernel Hilbert spaces (RKHSs) play an important role in many statistics and machine learning applications ranging from support vector machines to Gaussian processes and kernel embeddings of distributions. Operators acting on such spaces are, for instance, required to embed conditional probability distributions in order to implement the kernel Bayes rule and build sequential data models. It was recently shown that transfer operators such as the Perron-Frobenius or Koopman operator can also be approximated in a similar fashion using covariance and cross-covariance operators and that eigenfunctions of these operators can be obtained by solving associated matrix eigenvalue problems. The goal of this paper is to provide a solid functional analytic foundation for the eigenvalue decomposition of RKHS operators and to extend the approach to the singular value decomposition. The results are illustrated with simple guiding examples.