Unwrapping ADMM: Efficient Distributed Computing via Transpose Reduction

TL;DR

This paper introduces transpose reduction strategies enabling efficient distributed linear model fitting over massive datasets by solving global sub-problems, significantly reducing computation time compared to traditional consensus ADMM methods.

Contribution

It presents a novel iterative method that solves global least-squares problems in distributed settings, avoiding complex inner loops of consensus algorithms.

Findings

Successfully fitted models to 5 TB datasets using 7000 cores

Achieved faster computation times than previous methods

Demonstrated scalability and efficiency of the proposed approach

Abstract

Recent approaches to distributed model fitting rely heavily on consensus ADMM, where each node solves small sub-problems using only local data. We propose iterative methods that solve {\em global} sub-problems over an entire distributed dataset. This is possible using transpose reduction strategies that allow a single node to solve least-squares over massive datasets without putting all the data in one place. This results in simple iterative methods that avoid the expensive inner loops required for consensus methods. To demonstrate the efficiency of this approach, we fit linear classifiers and sparse linear models to datasets over 5 Tb in size using a distributed implementation with over 7000 cores in far less time than previous approaches.