Extreme AutoML: Analysis of Classification, Regression, and NLP Performance

TL;DR

This paper compares Extreme AutoML, based on Extreme Learning Machines, with Google AutoML across various datasets, demonstrating superior accuracy, efficiency, and stability for the former in classification tasks.

Contribution

It introduces and benchmarks Extreme AutoML, showing its advantages over existing AutoML solutions like Google AutoML in multiple performance metrics.

Findings

Extreme AutoML outperforms Google AutoML in accuracy.

Extreme AutoML has lower training times.

Extreme AutoML shows more consistent results across classes.

Abstract

Utilizing machine learning techniques has always required choosing hyperparameters. This is true whether one uses a classical technique such as a KNN or very modern neural networks such as Deep Learning. Though in many applications, hyperparameters are chosen by hand, automated methods have become increasingly more common. These automated methods have become collectively known as automated machine learning, or AutoML. Several automated selection algorithms have shown similar or improved performance over state-of-the-art methods. This breakthrough has led to the development of cloud-based services like Google AutoML, which is based on Deep Learning and is widely considered to be the industry leader in AutoML services. Extreme Learning Machines (ELMs) use a fundamentally different type of neural architecture, producing better results at a significantly discounted computational cost. We benchmark the Extreme AutoML technology against Google's AutoML using several popular classification data sets from the University of California at Irvine's (UCI) repository, and several other data sets, observing significant advantages for Extreme AutoML in accuracy, Jaccard Indices, the variance of Jaccard Indices across classes (i.e. class variance) and training times.