Efficient Deep Learning Board: Training Feedback Is Not All You Need

TL;DR

EfficientDL is a static deep learning performance prediction framework that recommends system components and predicts model performance without training feedback, significantly speeding up AutoML processes.

Contribution

It introduces a static prediction model and component recommendation algorithm that eliminate the need for training feedback in AutoDL, compatible with mainstream models.

Findings

Outperforms existing AutoML tools in accuracy and efficiency.

Operates seamlessly with popular DL models like ResNet50 and EfficientNet-B0.

Achieves approximately 20 times faster performance prediction with a 1.31% accuracy improvement.

Abstract

Current automatic deep learning (i.e., AutoDL) frameworks rely on training feedback from actual runs, which often hinder their ability to provide quick and clear performance predictions for selecting suitable DL systems. To address this issue, we propose EfficientDL, an innovative deep learning board designed for automatic performance prediction and component recommendation. EfficientDL can quickly and precisely recommend twenty-seven system components and predict the performance of DL models without requiring any training feedback. The magic of no training feedback comes from our proposed comprehensive, multi-dimensional, fine-grained system component dataset, which enables us to develop a static performance prediction model and comprehensive optimized component recommendation algorithm (i.e., {\alpha}\b{eta}-BO search), removing the dependency on actually running parameterized models during the traditional optimization search process. The simplicity and power of EfficientDL stem from its compatibility with most DL models. For example, EfficientDL operates seamlessly with mainstream models such as ResNet50, MobileNetV3, EfficientNet-B0, MaxViT-T, Swin-B, and DaViT-T, bringing competitive performance improvements. Besides, experimental results on the CIFAR-10 dataset reveal that EfficientDL outperforms existing AutoML tools in both accuracy and efficiency (approximately 20 times faster along with 1.31% Top-1 accuracy improvement than the cutting-edge methods). Source code, pretrained models, and datasets are available at https://github.com/OpenSELab/EfficientDL.