AEBNAS: Strengthening Exit Branches in Early-Exit Networks through Hardware-Aware Neural Architecture Search

TL;DR

This paper introduces a hardware-aware neural architecture search method to optimize exit branches in early-exit networks, improving accuracy and efficiency for resource-constrained devices.

Contribution

It proposes a novel NAS framework that considers layer depth and types in exit branches, enhancing early-exit network design with adaptive threshold tuning.

Findings

Achieves higher accuracy with similar or fewer MACs compared to state-of-the-art.

Demonstrates effectiveness on CIFAR-10, CIFAR-100, and SVHN datasets.

Optimizes exit branch design considering hardware constraints.

Abstract

Early-exit networks are effective solutions for reducing the overall energy consumption and latency of deep learning models by adjusting computation based on the complexity of input data. By incorporating intermediate exit branches into the architecture, they provide less computation for simpler samples, which is particularly beneficial for resource-constrained devices where energy consumption is crucial. However, designing early-exit networks is a challenging and time-consuming process due to the need to balance efficiency and performance. Recent works have utilized Neural Architecture Search (NAS) to design more efficient early-exit networks, aiming to reduce average latency while improving model accuracy by determining the best positions and number of exit branches in the architecture. Another important factor affecting the efficiency and accuracy of early-exit networks is the depth and types of layers in the exit branches. In this paper, we use hardware-aware NAS to strengthen exit branches, considering both accuracy and efficiency during optimization. Our performance evaluation on the CIFAR-10, CIFAR-100, and SVHN datasets demonstrates that our proposed framework, which considers varying depths and layers for exit branches along with adaptive threshold tuning, designs early-exit networks that achieve higher accuracy with the same or lower average number of MACs compared to the state-of-the-art approaches.