When Adaptive Diffusion Algorithm Converges to True Parameter?

TL;DR

This paper investigates the convergence conditions of adaptive diffusion algorithms, particularly diffusion RLS and RM, revealing limitations in high-dimensional cases and proposing conditions for successful convergence.

Contribution

It establishes convergence criteria for diffusion RLS with scalar parameters and introduces conditions under which diffusion RM guarantees convergence in high-dimensional settings.

Findings

Diffusion RLS converges for scalar parameters under specific conditions.

High-dimensional diffusion RLS may diverge despite local data consistency.

Diffusion RM achieves convergence under certain cooperative information conditions.

Abstract

We attempt to answer the question what data brings adaptive diffusion algorithms converging to true parameters. The discussion begins with the diffusion recursive least squares (RLS). When unknown parameters are scalar, the necessary and sufficient condition of the convergence for the diffusion RLS is established, in terms of the strong consistency and mean-square convergence both. However, for the general high dimensional parameter case, our results suggest that the diffusion RLS in a connected network might cause a diverging error, even if local data at every node could guarantee the individual RLS tending to true parameters. Due to the possible failure of the diffusion RLS, we prove that the diffusion Robbins-Monro (RM) algorithm could achieve the strong consistency and mean-square convergence simultaneously, under some cooperative information conditions. The convergence rates of the diffusion RM are derived explicitly.