A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters

TL;DR

This paper presents an AutoML workflow that efficiently builds diverse deep neural network ensembles and optimizes their inference on GPU clusters, balancing accuracy and computational cost.

Contribution

It introduces a new AutoML approach that constructs diverse models, employs a multi-objective ensemble selection, and optimizes GPU-based inference for DNN ensembles.

Findings

Asynchronous Hyperband effectively generates diverse models.

Multi-objective greedy algorithm improves ensemble accuracy and efficiency.

GPU cluster optimization enhances inference speed and resource utilization.

Abstract

Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.