Now that I can see, I can improve: Enabling data-driven finetuning of CNNs on the edge

TL;DR

This paper introduces a framework for fine-tuning CNNs directly on edge devices using structured pruning, enhancing accuracy while respecting privacy and resource constraints, demonstrated by significant performance gains.

Contribution

It presents the first approach for CNN fine-tuning on edge devices based on structured pruning, with an open-source framework adaptable to various architectures.

Findings

Data-aware pruning with retraining improves accuracy by 10.2 percentage points on average.

Maximum accuracy improvement of 42.0 percentage points achieved through pruning and retraining.

Framework enables deployment of CNN fine-tuning across diverse network architectures and devices.

Abstract

In today's world, a vast amount of data is being generated by edge devices that can be used as valuable training data to improve the performance of machine learning algorithms in terms of the achieved accuracy or to reduce the compute requirements of the model. However, due to user data privacy concerns as well as storage and communication bandwidth limitations, this data cannot be moved from the device to the data centre for further improvement of the model and subsequent deployment. As such there is a need for increased edge intelligence, where the deployed models can be fine-tuned on the edge, leading to improved accuracy and/or reducing the model's workload as well as its memory and power footprint. In the case of Convolutional Neural Networks (CNNs), both the weights of the network as well as its topology can be tuned to adapt to the data that it processes. This paper provides a first step towards enabling CNN finetuning on an edge device based on structured pruning. It explores the performance gains and costs of doing so and presents an extensible open-source framework that allows the deployment of such approaches on a wide range of network architectures and devices. The results show that on average, data-aware pruning with retraining can provide 10.2pp increased accuracy over a wide range of subsets, networks and pruning levels with a maximum improvement of 42.0pp over pruning and retraining in a manner agnostic to the data being processed by the network.