Spectral Eigenfunction Decomposition for Kernel Adaptive Filtering

TL;DR

This paper introduces SPEED, a spectral eigenfunction decomposition method that provides an explicit, low-dimensional Euclidean representation of the RKHS for kernel adaptive filtering, improving efficiency and scalability.

Contribution

The paper presents a novel spectral eigenfunction decomposition approach that efficiently approximates the kernel eigenspace for adaptive filtering, addressing scalability and dimensionality issues.

Findings

Outperforms existing explicit kernel feature methods in chaotic time series prediction

Enables dynamic tracking of kernel eigenspaces in adaptive filtering

Offers a scalable and efficient solution for kernel approximation

Abstract

Kernel adaptive filtering (KAF) integrates traditional linear algorithms with kernel methods to generate nonlinear solutions in the input space. The standard approach relies on the representer theorem and the kernel trick to perform pairwise evaluations of a kernel function in place of the inner product, which leads to scalability issues for large datasets due to its linear and superlinear growth with respect to the size of the training data. Explicit features have been proposed to tackle this problem, exploiting the properties of the Gaussian-type kernel functions. These approximation methods address the implicitness and infinite dimensional representation of conventional kernel methods. However, achieving an accurate finite approximation for the kernel evaluation requires a sufficiently large vector representation for the dot products. An increase in the input-space dimension leads to a combinatorial explosion in the dimensionality of the explicit space, i.e., it trades one dimensionality problem (implicit, infinite dimensional RKHS) for another (curse of dimensionality). This paper introduces a construction that simultaneously solves these two problems in a principled way, by providing an explicit Euclidean representation of the RKHS while reducing its dimensionality. We present SPEctral Eigenfunction Decomposition (SPEED) along with an efficient incremental approach for fast calculation of the dominant kernel eigenbasis, which enables us to track the kernel eigenspace dynamically for adaptive filtering. Simulation results on chaotic time series prediction demonstrate this novel construction outperforms existing explicit kernel features with greater efficiency.