Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark

TL;DR

This study evaluates whether AutoML frameworks can outperform human data scientists by comparing four AutoML tools on 12 popular datasets, finding that AutoML performs better or equally in over half of the tasks.

Contribution

The paper provides a comprehensive comparison of AutoML frameworks against human performance on diverse datasets, highlighting AutoML's competitive capabilities.

Findings

AutoML outperforms or matches human results in 7 of 12 datasets

AutoML performs well on both classification and regression tasks

AutoML shows promise for real-world applications

Abstract

In the last few years, Automated Machine Learning (AutoML) has gained much attention. With that said, the question arises whether AutoML can outperform results achieved by human data scientists. This paper compares four AutoML frameworks on 12 different popular datasets from OpenML; six of them supervised classification tasks and the other six supervised regression ones. Additionally, we consider a real-life dataset from one of our recent projects. The results show that the automated frameworks perform better or equal than the machine learning community in 7 out of 12 OpenML tasks.