Tune: A Research Platform for Distributed Model Selection and Training

TL;DR

Tune is a versatile platform that streamlines distributed hyperparameter search and model training, enabling scalable, efficient, and easy-to-implement algorithms across large compute clusters.

Contribution

It introduces a unified framework with a standardized interface that simplifies the development and scaling of hyperparameter search algorithms in distributed environments.

Findings

Supports a broad range of hyperparameter search algorithms

Enables straightforward scaling to large clusters

Simplifies implementation of search algorithms

Abstract

Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adaptation to the distributed compute environment is often ad-hoc. We propose Tune, a unified framework for model selection and training that provides a narrow-waist interface between training scripts and search algorithms. We show that this interface meets the requirements for a broad range of hyperparameter search algorithms, allows straightforward scaling of search to large clusters, and simplifies algorithm implementation. We demonstrate the implementation of several state-of-the-art hyperparameter search algorithms in Tune. Tune is available at http://ray.readthedocs.io/en/latest/tune.html.