Automated Machine Learning with Monte-Carlo Tree Search

TL;DR

This paper introduces Mosaic, an AutoML approach using Monte-Carlo tree search to optimize algorithm selection and hyperparameters, demonstrating significant improvements over existing methods on benchmark datasets.

Contribution

The paper presents Mosaic, a novel MCTS-based AutoML method that effectively handles hybrid black-box optimization problems, outperforming Bayesian optimization approaches.

Findings

Mosaic achieves statistically significant performance gains.

MCTS-based AutoML outperforms Bayesian optimization in experiments.

Ensembling solutions improves AutoML results.

Abstract

The AutoML task consists of selecting the proper algorithm in a machine learning portfolio, and its hyperparameter values, in order to deliver the best performance on the dataset at hand. Mosaic, a Monte-Carlo tree search (MCTS) based approach, is presented to handle the AutoML hybrid structural and parametric expensive black-box optimization problem. Extensive empirical studies are conducted to independently assess and compare: i) the optimization processes based on Bayesian optimization or MCTS; ii) its warm-start initialization; iii) the ensembling of the solutions gathered along the search. Mosaic is assessed on the OpenML 100 benchmark and the Scikit-learn portfolio, with statistically significant gains over Auto-Sklearn, winner of former international AutoML challenges.