Fast Learning in Reproducing Kernel Krein Spaces via Signed Measures

TL;DR

This paper introduces a distribution-based framework for decomposing non-positive definite kernels into the difference of two positive definite kernels, enabling scalable learning with unbiased random features.

Contribution

It provides the first necessary and sufficient condition for positive decomposition of non-PD kernels using signed measures, along with a practical algorithm for large-scale applications.

Findings

The proposed algorithm effectively scales non-PD kernels to large datasets.

Experimental results show improved performance over existing methods.

The framework offers a new theoretical understanding of non-PD kernel decompositions.

Abstract

In this paper, we attempt to solve a long-lasting open question for non-positive definite (non-PD) kernels in machine learning community: can a given non-PD kernel be decomposed into the difference of two PD kernels (termed as positive decomposition)? We cast this question as a distribution view by introducing the \emph{signed measure}, which transforms positive decomposition to measure decomposition: a series of non-PD kernels can be associated with the linear combination of specific finite Borel measures. In this manner, our distribution-based framework provides a sufficient and necessary condition to answer this open question. Specifically, this solution is also computationally implementable in practice to scale non-PD kernels in large sample cases, which allows us to devise the first random features algorithm to obtain an unbiased estimator. Experimental results on several benchmark datasets verify the effectiveness of our algorithm over the existing methods.