Q-linear Convergence of Distributed Optimization with Barzilai-Borwein Step Sizes

TL;DR

This paper introduces a distributed quasi-Newton optimization method using Barzilai-Borwein step sizes, achieving Q-linear convergence and potential superlinear convergence, validated through numerical experiments.

Contribution

It proposes a novel distributed quasi-Newton method with Barzilai-Borwein step sizes, demonstrating convergence properties and practical effectiveness.

Findings

Proves Q-linear convergence to the optimal solution.

Identifies conditions for superlinear convergence.

Validates results with numerical simulations.

Abstract

The growth in sizes of large-scale systems and data in machine learning have made distributed optimization a naturally appealing technique to solve decision problems in different contexts. In such methods, each agent iteratively carries out computations on its local objective using information received from its neighbors, and shares relevant information with neighboring agents. Though gradient-based methods are widely used because of their simplicity, they are known to have slow convergence rates. On the other hand, though Newton-type methods have better convergence properties, they are not as applicable because of the enormous computation and memory requirements. In this work, we introduce a distributed quasi-Newton method with Barzilai-Borwein step-sizes. We prove a Q-linear convergence to the optimal solution, present conditions under which the algorithm is superlinearly convergent and validate our results via numerical simulations.