Constrained Multi-Objective Optimization for Automated Machine Learning

TL;DR

The paper introduces Autotune, a framework for multi-objective and constrained automated machine learning, leveraging derivative-free optimization and parallelism to efficiently explore trade-offs and improve model selection.

Contribution

It presents Autotune, a novel framework that effectively handles multiple objectives and constraints in automated machine learning using derivative-free methods and distributed computing.

Findings

Autotune efficiently captures Pareto fronts in benchmark problems.

Adding constraints guides the search to more promising solutions.

Autotune demonstrates effectiveness in real-world case studies.

Abstract

Automated machine learning has gained a lot of attention recently. Building and selecting the right machine learning models is often a multi-objective optimization problem. General purpose machine learning software that simultaneously supports multiple objectives and constraints is scant, though the potential benefits are great. In this work, we present a framework called Autotune that effectively handles multiple objectives and constraints that arise in machine learning problems. Autotune is built on a suite of derivative-free optimization methods, and utilizes multi-level parallelism in a distributed computing environment for automatically training, scoring, and selecting good models. Incorporation of multiple objectives and constraints in the model exploration and selection process provides the flexibility needed to satisfy trade-offs necessary in practical machine learning applications. Experimental results from standard multi-objective optimization benchmark problems show that Autotune is very efficient in capturing Pareto fronts. These benchmark results also show how adding constraints can guide the search to more promising regions of the solution space, ultimately producing more desirable Pareto fronts. Results from two real-world case studies demonstrate the effectiveness of the constrained multi-objective optimization capability offered by Autotune.