Automated Machine Learning -- a brief review at the end of the early years

TL;DR

This paper reviews the early development of AutoML, summarizing key achievements, paradigms, and future research directions in automating machine learning system design for supervised learning.

Contribution

It provides a comprehensive overview of AutoML's progress during its initial years, highlighting main paradigms and outlining future research opportunities.

Findings

AutoML has significantly advanced in the past decade.

Main paradigms of AutoML include hyperparameter optimization and neural architecture search.

AutoML research has identified key challenges and future directions.

Abstract

Automated machine learning (AutoML) is the sub-field of machine learning that aims at automating, to some extend, all stages of the design of a machine learning system. In the context of supervised learning, AutoML is concerned with feature extraction, pre processing, model design and post processing. Major contributions and achievements in AutoML have been taking place during the recent decade. We are therefore in perfect timing to look back and realize what we have learned. This chapter aims to summarize the main findings in the early years of AutoML. More specifically, in this chapter an introduction to AutoML for supervised learning is provided and an historical review of progress in this field is presented. Likewise, the main paradigms of AutoML are described and research opportunities are outlined.