MANGO: A Python Library for Parallel Hyperparameter Tuning

TL;DR

MANGO is an open-source Python library that facilitates flexible, fault-tolerant, and scalable parallel hyperparameter tuning across various distributed computing frameworks, improving automation in machine learning workflows.

Contribution

MANGO introduces a flexible, framework-agnostic library with advanced search strategies and abstractions for complex hyperparameter spaces, addressing gaps in existing tuning tools.

Findings

Comparable performance to Hyperopt

Supports any distributed scheduling framework

Used in production at Arm Research

Abstract

Tuning hyperparameters for machine learning algorithms is a tedious task, one that is typically done manually. To enable automated hyperparameter tuning, recent works have started to use techniques based on Bayesian optimization. However, to practically enable automated tuning for large scale machine learning training pipelines, significant gaps remain in existing libraries, including lack of abstractions, fault tolerance, and flexibility to support scheduling on any distributed computing framework. To address these challenges, we present Mango, a Python library for parallel hyperparameter tuning. Mango enables the use of any distributed scheduling framework, implements intelligent parallel search strategies, and provides rich abstractions for defining complex hyperparameter search spaces that are compatible with scikit-learn. Mango is comparable in performance to Hyperopt, another widely used library. Mango is available open-source and is currently used in production at Arm Research to provide state-of-art hyperparameter tuning capabilities.