Generalized ADMM in Distributed Learning via Variational Inequality

TL;DR

This paper unifies various ADMM-based distributed learning algorithms using variational inequality, demonstrating how flexible consensus parameters and uncertainty modeling improve learning performance through simulations.

Contribution

It introduces a VI-based framework to analyze and generalize ADMM variations in distributed learning, enhancing understanding and performance.

Findings

Flexible consensus parameters improve convergence.

Uncertain parameters can enhance learning outcomes.

VI framework unifies different ADMM approaches.

Abstract

Due to the explosion in size and complexity of modern data sets and privacy concerns of data holders, it is increasingly important to be able to solve machine learning problems in distributed manners. The Alternating Direction Method of Multipliers (ADMM) through the concept of consensus variables is a practical algorithm in this context where its diverse variations and its performance have been studied in different application areas. In this paper, we study the effect of the local data sets of users in the distributed learning of ADMM. Our aim is to deploy variational inequality (VI) to attain an unified view of ADMM variations. Through the simulation results, we demonstrate how more general definitions of consensus parameters and introducing the uncertain parameters in distribute approach can help to get the better results in learning processes.