A Unified Algorithm for Nonconvex Decentralized Nonlinear Optimization

TL;DR

This paper introduces a unified decentralized algorithmic framework for nonconvex optimization over networks, encompassing existing methods and proposing new quasi-Newton algorithms with proven convergence.

Contribution

The paper develops a general framework for decentralized nonconvex optimization, including new quasi-Newton algorithms and convergence analysis under broad conditions.

Findings

New algorithms outperform existing methods in numerical tests.

Unified framework covers many state-of-the-art decentralized algorithms.

Convergence proven under nonconvex and Kurdyka-Łojasiewicz conditions.

Abstract

In this paper, we study the decentralized optimization problem of minimizing a finite sum of continuously differentiable and possibly nonconvex functions over a fixed-connected undirected network. We propose a unified decentralized nonconvex algorithmic framework that includes many existing state-of-the-art gradient tracking and quasi-Newton algorithms. A general framework for the convergence analysis of our unified algorithm is presented under both nonconvex and the Kurdyka-{\L}ojasiewicz condition settings. In particular, some new quasi-Newton algorithms under this framework are proposed. Our numerical results show that these newly developed algorithms are very efficient compared with other state-of-the-art algorithms for solving decentralized nonconvex nonlinear optimization.