CompactNet: Platform-Aware Automatic Optimization for Convolutional Neural Networks

TL;DR

CompactNet is an automated platform-aware optimization tool that trims pre-trained CNNs to meet specific speed and resource constraints while preserving accuracy, demonstrated on mobile and NPU platforms.

Contribution

It introduces a novel method for automatic, platform-specific CNN model optimization guided by a simulator, enabling efficient deployment on resource-limited devices.

Findings

Achieves up to 1.8x speedup on MobileNetV2

Maintains or improves accuracy after optimization

Effective on ARM CPU and NPU platforms

Abstract

Convolutional Neural Network (CNN) based Deep Learning (DL) has achieved great progress in many real-life applications. Meanwhile, due to the complex model structures against strict latency and memory restriction, the implementation of CNN models on the resource-limited platforms is becoming more challenging. This work proposes a solution, called CompactNet\footnote{Project URL: \url{https://github.com/CompactNet/CompactNet}}, which automatically optimizes a pre-trained CNN model on a specific resource-limited platform given a specific target of inference speedup. Guided by a simulator of the target platform, CompactNet progressively trims a pre-trained network by removing certain redundant filters until the target speedup is reached and generates an optimal platform-specific model while maintaining the accuracy. We evaluate our work on two platforms of a mobile ARM CPU and a machine learning accelerator NPU (Cambricon-1A ISA) on a Huawei Mate10 smartphone. For the state-of-the-art slim CNN model made for the embedded platform, MobileNetV2, CompactNet achieves up to a 1.8x kernel computation speedup with equal or even higher accuracy for image classification tasks on the Cifar-10 dataset.