Differentiable Network Pruning for Microcontrollers

TL;DR

This paper introduces a differentiable structured network pruning method tailored for microcontroller units, significantly reducing model size and resource usage while maintaining or improving accuracy, suitable for resource-constrained IoT devices.

Contribution

It presents a novel MCU-specific network pruning technique that integrates resource and importance feedback, achieving high compression with minimal training overhead.

Findings

Up to 80x resource usage improvement

Pruning during training with little overhead

Models compressed up to 1.4x faster than prior methods

Abstract

Embedded and personal IoT devices are powered by microcontroller units (MCUs), whose extreme resource scarcity is a major obstacle for applications relying on on-device deep learning inference. Orders of magnitude less storage, memory and computational capacity, compared to what is typically required to execute neural networks, impose strict structural constraints on the network architecture and call for specialist model compression methodology. In this work, we present a differentiable structured network pruning method for convolutional neural networks, which integrates a model's MCU-specific resource usage and parameter importance feedback to obtain highly compressed yet accurate classification models. Our methodology (a) improves key resource usage of models up to 80x; (b) prunes iteratively while a model is trained, resulting in little to no overhead or even improved training time; (c) produces compressed models with matching or improved resource usage up to 1.4x in less time compared to prior MCU-specific methods. Compressed models are available for download.