Distributed Conjugate Gradient Method via Conjugate Direction Tracking

TL;DR

This paper introduces a distributed conjugate gradient algorithm that allows agents to optimize collectively without central coordination, using local computations and neighbor communications, with proven convergence and demonstrated effectiveness.

Contribution

The paper proposes a novel distributed conjugate gradient method utilizing conjugate direction tracking and dynamic consensus, enabling uncoordinated step-sizes and proven convergence.

Findings

Algorithm converges to the optimal solution without decreasing step-sizes.

Effective in distributed state estimation and robust variants.

Outperforms existing distributed first-order methods.

Abstract

We present a distributed conjugate gradient method for distributed optimization problems, where each agent computes an optimal solution of the problem locally without any central computation or coordination, while communicating with its immediate, one-hop neighbors over a communication network. Each agent updates its local problem variable using an estimate of the average conjugate direction across the network, computed via a dynamic consensus approach. Our algorithm enables the agents to use uncoordinated step-sizes. We prove convergence of the local variable of each agent to the optimal solution of the aggregate optimization problem, without requiring decreasing step-sizes. In addition, we demonstrate the efficacy of our algorithm in distributed state estimation problems, and its robust counterparts, where we show its performance compared to existing distributed first-order optimization methods.