NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion

TL;DR

NOMAD is a decentralized, lock-free, asynchronous algorithm for matrix completion that efficiently transfers variable ownership between processors, outperforming synchronous methods in distributed and HPC environments.

Contribution

The paper introduces NOMAD, a novel lock-free, asynchronous, decentralized matrix completion algorithm with serializable updates and superior performance over existing methods.

Findings

NOMAD achieves faster convergence than synchronous algorithms.

NOMAD performs well on commodity hardware and HPC clusters.

The algorithm outperforms state-of-the-art methods in distributed settings.

Abstract

We develop an efficient parallel distributed algorithm for matrix completion, named NOMAD (Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion). NOMAD is a decentralized algorithm with non-blocking communication between processors. One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion. As a consequence it is a lock-free parallel algorithm. In spite of being an asynchronous algorithm, the variable updates of NOMAD are serializable, that is, there is an equivalent update ordering in a serial implementation. NOMAD outperforms synchronous algorithms which require explicit bulk synchronization after every iteration: our extensive empirical evaluation shows that not only does our algorithm perform well in distributed setting on commodity hardware, but also outperforms state-of-the-art algorithms on a HPC cluster both in multi-core and distributed memory settings.