Knowledge-Aided Kaczmarz and LMS Algorithms

TL;DR

This paper introduces Knowledge-Aided Kaczmarz and LMS algorithms that incorporate prior information into system identification, improving performance especially in low SNR scenarios and approaching MAP optimality.

Contribution

It develops new algorithms based on maximum a posteriori principles that integrate prior mean and covariance, enhancing traditional LMS and Kaczmarz methods.

Findings

Algorithms improve estimation accuracy with reliable priors.

Performance approaches MAP optimality in low SNR conditions.

Convergence analysis confirms stability of the proposed methods.

Abstract

The least mean squares (LMS) filter is often derived via the Wiener filter solution. For a system identification scenario, such a derivation makes it hard to incorporate prior information on the system's impulse response. We present an alternative way based on the maximum a posteriori solution, which allows developing a Knowledge-Aided Kaczmarz algorithm. Based on this Knowledge-Aided Kaczmarz we formulate a Knowledge-Aided LMS filter. Both algorithms allow incorporating the prior mean and covariance matrix on the parameter to be estimated. The algorithms use this prior information in addition to the measurement information in the gradient for the iterative update of their estimates. We analyze the convergence of the algorithms and show simulation results on their performance. As expected, reliable prior information allows improving the performance of the algorithms for low signal-to-noise (SNR) scenarios. The results show that the presented algorithms can nearly achieve the optimal maximum a posteriori (MAP) performance.