Gramian-Based Adaptive Combination Policies for Diffusion Learning over Networks

TL;DR

This paper introduces a novel adaptive combination policy for diffusion networks that improves transient learning performance by dynamically adjusting weights based on data quality, enhancing overall distributed learning.

Contribution

It develops a new adaptive combination rule focused on optimizing transient behavior while preserving steady-state performance in diffusion learning networks.

Findings

The proposed policy improves transient learning performance.

It maintains steady-state performance comparable to existing methods.

The approach adapts weights based on data quality for better collaboration.

Abstract

This paper presents an adaptive combination strategy for distributed learning over diffusion networks. Since learning relies on the collaborative processing of the stochastic information at the dispersed agents, the overall performance can be improved by designing combination policies that adjust the weights according to the quality of the data. Such policies are important because they would add a new degree of freedom and endow multi-agent systems with the ability to control the flow of information over their edges for enhanced performance. Most adaptive and static policies available in the literature optimize certain performance metrics related to steady-state behavior, to the detriment of transient behavior. In contrast, we develop an adaptive combination rule that aims at optimizing the transient learning performance, while maintaining the enhanced steady-state performance obtained using policies previously developed in the literature.