Improving Sparsity in Kernel Adaptive Filters Using a Unit-Norm Dictionary

TL;DR

This paper introduces a novel unit-norm Gaussian kernel for kernel adaptive filters, reducing dictionary growth and improving prediction accuracy for monotonic signals by normalizing observations before comparison.

Contribution

It proposes a new unit-norm Gaussian kernel and sparsification criterion, enhancing sparsity and predictive performance in kernel adaptive filtering.

Findings

Reduces dictionary size compared to standard KAF.

Improves normalized mean square error on real datasets.

Effectively handles monotonic signals with fewer samples.

Abstract

Kernel adaptive filters, a class of adaptive nonlinear time-series models, are known by their ability to learn expressive autoregressive patterns from sequential data. However, for trivial monotonic signals, they struggle to perform accurate predictions and at the same time keep computational complexity within desired boundaries. This is because new observations are incorporated to the dictionary when they are far from what the algorithm has seen in the past. We propose a novel approach to kernel adaptive filtering that compares new observations against dictionary samples in terms of their unit-norm (normalised) versions, meaning that new observations that look like previous samples but have a different magnitude are not added to the dictionary. We achieve this by proposing the unit-norm Gaussian kernel and define a sparsification criterion for this novel kernel. This new methodology is validated on two real-world datasets against standard KAF in terms of the normalised mean square error and the dictionary size.