A stochastic behavior analysis of stochastic restricted-gradient descent algorithm in reproducing kernel Hilbert spaces

TL;DR

This paper analyzes the stochastic behavior and stability of a kernel-based restricted-gradient descent algorithm in reproducing kernel Hilbert spaces, providing insights into its transient and steady-state performance.

Contribution

It offers a novel stochastic analysis of the restricted-gradient descent method in RKHS, including stability conditions and performance evaluation.

Findings

Provides stability conditions in mean and mean-square sense

Derives transient and steady-state mean squared error performance

Simulation results validate the theoretical analysis

Abstract

This paper presents a stochastic behavior analysis of a kernel-based stochastic restricted-gradient descent method. The restricted gradient gives a steepest ascent direction within the so-called dictionary subspace. The analysis provides the transient and steady state performance in the mean squared error criterion. It also includes stability conditions in the mean and mean-square sense. The present study is based on the analysis of the kernel normalized least mean square (KNLMS) algorithm initially proposed by Chen et al. Simulation results validate the analysis.