Bundle EXTRA for Decentralized Optimization

TL;DR

This paper introduces a fully decentralized bundle EXTRA method that replaces first-order approximations with a multi-cut bundle model, improving convergence speed and robustness in decentralized optimization.

Contribution

It proposes a novel bundle-based primal update for decentralized primal-dual methods, enhancing convergence and robustness over existing approaches.

Findings

Bundle EXTRA converges faster than traditional EXTRA.

The method is more robust to step-size variations.

Numerical experiments confirm improved performance on least-squares problems.

Abstract

Decentralized primal-dual methods are widely used for solving decentralized optimization problems, but their updates often rely on the potentially crude first-order Taylor approximations of the objective functions, which can limit convergence speed. To overcome this, we replace the first-order Taylor approximation in the primal update of EXTRA, which can be interpreted as a primal-dual method, with a more accurate multi-cut bundle model, resulting in a fully decentralized bundle EXTRA method. The bundle model incorporates historical information to improve the approximation accuracy, potentially leading to faster convergence. Under mild assumptions, we show that a KKT residual converges to zero. Numerical experiments on decentralized least-squares problems demonstrate that, compared to EXTRA, the bundle EXTRA method converges faster and is more robust to step-size choices.