Kernel Recursive Least Squares Dictionary Learning Algorithm

TL;DR

This paper introduces an efficient online kernel dictionary learning algorithm that updates a virtual dictionary recursively, enabling high-dimensional sparse representations with low computational cost, and demonstrates superior performance across multiple datasets.

Contribution

It presents a novel recursive least squares-based online kernel dictionary learning method that improves efficiency and accuracy over existing approaches.

Findings

Outperforms existing online kernel dictionary learning methods

Achieves classification accuracy close to batch models

Maintains low computational complexity

Abstract

We propose an efficient online dictionary learning algorithm for kernel-based sparse representations. In this framework, input signals are nonlinearly mapped to a high-dimensional feature space and represented sparsely using a virtual dictionary. At each step, the dictionary is updated recursively using a novel algorithm based on the recursive least squares (RLS) method. This update mechanism works with single samples or mini-batches and maintains low computational complexity. Experiments on four datasets across different domains show that our method not only outperforms existing online kernel dictionary learning approaches but also achieves classification accuracy close to that of batch-trained models, while remaining significantly more efficient.