Autostacker: A Compositional Evolutionary Learning System

TL;DR

Autostacker is an AutoML system that uses a hierarchical stacking architecture and evolutionary algorithms to automatically generate high-performing machine learning pipelines without prior domain knowledge.

Contribution

It introduces a novel AutoML architecture combining hierarchical stacking with evolutionary algorithms for efficient pipeline search.

Findings

Achieves state-of-the-art or competitive accuracy on multiple datasets.

Reduces time cost compared to existing AutoML systems.

Automatically discovers innovative model combinations.

Abstract

We introduce an automatic machine learning (AutoML) modeling architecture called Autostacker, which combines an innovative hierarchical stacking architecture and an Evolutionary Algorithm (EA) to perform efficient parameter search. Neither prior domain knowledge about the data nor feature preprocessing is needed. Using EA, Autostacker quickly evolves candidate pipelines with high predictive accuracy. These pipelines can be used as is or as a starting point for human experts to build on. Autostacker finds innovative combinations and structures of machine learning models, rather than selecting a single model and optimizing its hyperparameters. Compared with other AutoML systems on fifteen datasets, Autostacker achieves state-of-art or competitive performance both in terms of test accuracy and time cost.