Tell Me Something New: A New Framework for Asynchronous Parallel Learning

TL;DR

The paper introduces TMSN, a new asynchronous parallel learning framework that enables independent workers to share new information without synchronization, improving speed and resilience in machine learning tasks.

Contribution

It proposes the TMSN framework for asynchronous parallel learning, demonstrating significant speed improvements and robustness over existing methods like XGBoost and LightGBM.

Findings

TMSN is 10 times faster than XGBoost and LightGBM on splice-site prediction.

TMSN operates without synchronization or a central coordinator.

The framework is highly resilient to machine failures and delays.

Abstract

We present a novel approach for parallel computation in the context of machine learning that we call "Tell Me Something New" (TMSN). This approach involves a set of independent workers that use broadcast to update each other when they observe "something new". TMSN does not require synchronization or a head node and is highly resilient against failing machines or laggards. We demonstrate the utility of TMSN by applying it to learning boosted trees. We show that our implementation is 10 times faster than XGBoost and LightGBM on the splice-site prediction problem.