Projection-based Prediction-Correction Method for Distributed Consensus Optimization

TL;DR

This paper introduces a novel adaptive projection prediction-correction method (PPCM) for distributed consensus optimization, demonstrating superior speed and accuracy in large-scale networked systems across various applications.

Contribution

The paper presents a new PPCM algorithm inspired by proximal point methods and variational inequalities, with simple parameter tuning and proven convergence for decentralized networks.

Findings

PPCM achieves over ten times faster computation than Python built-in functions.

The method maintains high precision in distributed linear least squares, logistic regression, and SVMs.

Theoretical analysis confirms convergence and efficiency of PPCM.

Abstract

Within the realm of industrial technology, optimization methods play a pivotal role and are extensively applied across various sectors, including transportation engineering, robotics, and machine learning. With the surge in data volumes, there is an increasing demand for solving large-scale problems, which in turn has spurred the development of distributed optimization methods. These methods rely on the collaborative efforts of numerous dispersed devices to achieve the collective goals of the system. This study focuses on the exploration of distributed consensus optimization problems with convex set constraints within networks. The paper introduces a novel Adaptive Projection Prediction-Correction Method (PPCM), inspired by the proximal point algorithm and incorporating the theory of variational inequalities. As a contraction algorithm with notable convergence performance, PPCM is particularly suited for decentralized network environments. Moreover, the selection of parameters for this method is both straightforward and intuitive, avoiding the complexities of intricate parameter tuning. Comprehensive theoretical analysis and empirical testing have validated the effectiveness of PPCM. When applied to problems such as distributed linear least squares, logistic regression, and support vector machines, PPCM demonstrates superior performance, achieving computation speeds over ten times faster than built-in Python functions while maintaining high precision. In conclusion, this research provides a valuable distributed consensus optimization technique, both theoretically and practically.