Analysis via Orthonormal Systems in Reproducing Kernel Hilbert $C^*$-Modules and Applications

TL;DR

This paper introduces a new framework using reproducing kernel Hilbert $C^*$-modules for data analysis, extending kernel methods to handle more complex variable structures and applying it to PCA and dynamical systems.

Contribution

It develops a theoretical foundation for orthonormal systems in Hilbert $C^*$-modules and proposes practical procedures for their orthonormalization in data analysis.

Findings

Theoretical validation of orthonormal systems in Hilbert $C^*$-modules.

Generalization of kernel PCA using RKHMs.

Empirical evaluation on synthetic and real-world datasets.

Abstract

Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert $C^*$-module (RKHM), which is another generalization of RKHS than vector-valued RKHS (vv-RKHS). Analysis with RKHMs enables us to deal with structures among variables more explicitly than vv-RKHS. We show the theoretical validity for the construction of orthonormal systems in Hilbert $C^*$-modules, and derive concrete procedures for orthonormalization in RKHMs with those theoretical properties in numerical computations. Moreover, we apply those to generalize with RKHM kernel principal component analysis and the analysis of dynamical systems with Perron-Frobenius operators. The empirical performance of our methods is also investigated by using synthetic and real-world data.