SqueezeNext: Hardware-Aware Neural Network Design

TL;DR

SqueezeNext is a family of neural networks optimized for embedded systems, achieving high accuracy with significantly fewer parameters and better efficiency, guided by hardware-aware design and simulation results.

Contribution

Introduces SqueezeNext, a neural network architecture designed with hardware considerations, offering high accuracy with fewer parameters and improved efficiency over prior models.

Findings

Matches AlexNet accuracy with 112x fewer parameters.

Achieves VGG-19 accuracy with only 4.4 million parameters.

Designs are 2.59x/8.26x faster and 2.25x/7.5x more energy efficient than SqueezeNet/AlexNet.

Abstract

One of the main barriers for deploying neural networks on embedded systems has been large memory and power consumption of existing neural networks. In this work, we introduce SqueezeNext, a new family of neural network architectures whose design was guided by considering previous architectures such as SqueezeNet, as well as by simulation results on a neural network accelerator. This new network is able to match AlexNet's accuracy on the ImageNet benchmark with $112\times$ fewer parameters, and one of its deeper variants is able to achieve VGG-19 accuracy with only 4.4 Million parameters, ($31\times$ smaller than VGG-19). SqueezeNext also achieves better top-5 classification accuracy with $1.3\times$ fewer parameters as compared to MobileNet, but avoids using depthwise-separable convolutions that are inefficient on some mobile processor platforms. This wide range of accuracy gives the user the ability to make speed-accuracy tradeoffs, depending on the available resources on the target hardware. Using hardware simulation results for power and inference speed on an embedded system has guided us to design variations of the baseline model that are $2.59\times$/$8.26\times$ faster and $2.25\times$/$7.5\times$ more energy efficient as compared to SqueezeNet/AlexNet without any accuracy degradation.