DJAM: distributed Jacobi asynchronous method for learning personal models

TL;DR

DJAM is a distributed, asynchronous algorithm for learning personalized models in networked data, converging reliably without hyperparameter tuning and performing comparably to state-of-the-art methods.

Contribution

Introduces DJAM, a hyperparameter-free, asynchronous distributed algorithm for personalized model learning with convergence guarantees.

Findings

DJAM converges with probability one under strong convexity and Lipschitz conditions.

DJAM achieves similar accuracy to tuned ADMM in comparable interaction counts.

The method requires only single-neighbor interactions, simplifying implementation.

Abstract

Processing data collected by a network of agents often boils down to solving an optimization problem. The distributed nature of these problems calls for methods that are, themselves, distributed. While most collaborative learning problems require agents to reach a common (or consensus) model, there are situations in which the consensus solution may not be optimal. For instance, agents may want to reach a compromise between agreeing with their neighbors and minimizing a personal loss function. We present DJAM, a Jacobi-like distributed algorithm for learning personalized models. This method is implementation-friendly: it has no hyperparameters that need tuning, it is asynchronous, and its updates only require single-neighbor interactions. We prove that DJAM converges with probability one to the solution, provided that the personal loss functions are strongly convex and have Lipschitz gradient. We then give evidence that DJAM is on par with state-of-the-art methods: our method reaches a solution with error similar to the error of a carefully tuned ADMM in about the same number of single-neighbor interactions.