Fast model averaging via buffered states and first-order accelerated optimization algorithms

TL;DR

This paper introduces two buffered state-based methods to accelerate average consensus in networks without changing the graph structure, leveraging convex optimization and momentum algorithms for faster convergence.

Contribution

It proposes novel buffered state techniques and an accelerated optimization approach for consensus, improving convergence speed without altering network connectivity.

Findings

Buffered states can significantly speed up consensus convergence.

Accelerated algorithms outperform traditional methods in GMM estimation.

Less global knowledge is needed for the first approach.

Abstract

In this letter, we study the problem of accelerating reaching average consensus over connected graphs in a discrete-time communication setting. Literature has shown that consensus algorithms can be accelerated by increasing the graph connectivity or optimizing the weights agents place on the information received from their neighbors. Here, instead of altering the communication graph, we investigate two methods that use buffered states to accelerate reaching average consensus over a given graph. In the first method, we study how convergence rate of the well-known first-order Laplacian average consensus algorithm changes when agreement feedback is generated from buffered states. For this study, we obtain a sufficient condition on the ranges of buffered state that leads to faster convergence. In the second proposed method, we show how the average consensus problem can be cast as a convex optimization problem and solved by first-order accelerated optimization algorithms for strongly-convex cost functions. We construct an accelerated average consensus algorithm using the so-called Triple Momentum optimization algorithm. The first approach requires less global knowledge for choosing the step size, whereas the second one converges faster in our numerical results by using extra information from the graph topology. We demonstrate our results by implementing the proposed algorithms in a Gaussian Mixture Model (GMM) estimation problem used in sensor networks.