COLA: Decentralized Linear Learning

TL;DR

COLA is a decentralized training algorithm for linear models that ensures privacy, efficiency, scalability, and robustness in a network of user devices without central coordination.

Contribution

We introduce COLA, a novel decentralized learning algorithm with strong theoretical guarantees and improved practical performance for on-device linear model training.

Findings

Achieves communication efficiency and scalability.

Demonstrates resilience to data and device changes.

Outperforms existing decentralized learning methods.

Abstract

Decentralized machine learning is a promising emerging paradigm in view of global challenges of data ownership and privacy. We consider learning of linear classification and regression models, in the setting where the training data is decentralized over many user devices, and the learning algorithm must run on-device, on an arbitrary communication network, without a central coordinator. We propose COLA, a new decentralized training algorithm with strong theoretical guarantees and superior practical performance. Our framework overcomes many limitations of existing methods, and achieves communication efficiency, scalability, elasticity as well as resilience to changes in data and participating devices.