dHPR: A Distributed Halpern Peaceman--Rachford Method for Non-smooth Distributed Optimization Problems

TL;DR

This paper presents dHPR, a novel distributed optimization algorithm that efficiently handles non-smooth convex problems with proven convergence rates and is validated through extensive numerical experiments.

Contribution

The paper introduces the dHPR method, combining symmetric Gauss--Seidel decomposition with a decentralized approach for non-smooth convex optimization, achieving fast convergence without large proximal terms.

Findings

Achieves non-ergodic $O(1/k)$ convergence rate.

Effectively decouples operators for parallel implementation.

Demonstrates superior performance on LASSO and logistic regression.

Abstract

This paper introduces the distributed Halpern Peaceman--Rachford (dHPR) method, an efficient algorithm for solving distributed convex composite optimization problems with non-smooth objectives, which achieves a non-ergodic $O(1/k)$ iteration complexity regarding Karush--Kuhn--Tucker residual. By leveraging the symmetric Gauss--Seidel decomposition, the dHPR effectively decouples the linear operators in the objective functions and consensus constraints while maintaining parallelizability and avoiding additional large proximal terms, leading to a decentralized implementation with provably fast convergence. The superior performance of dHPR is demonstrated through comprehensive numerical experiments on distributed LASSO, group LASSO, and $L_1$-regularized logistic regression problems.