DANCE: Dynamic 3D CNN Pruning: Joint Frame, Channel, and Feature Adaptation for Energy Efficiency on the Edge

TL;DR

DANCE is a dynamic pruning framework for 3D CNNs that adaptively reduces computation and energy consumption by pruning frames, channels, and features based on input variability, with minimal performance loss.

Contribution

The paper introduces a novel two-step input-aware dynamic pruning method for 3D CNNs, enhancing energy efficiency and computational savings on edge devices.

Findings

Achieves up to 1.47X higher energy efficiency.

Demonstrates 1.37X and 2.22X speedups on NVIDIA Jetson Nano and Snapdragon platforms.

Reduces multiply-accumulate operations and memory accesses significantly.

Abstract

Modern convolutional neural networks (CNNs) are workhorses for video and image processing, but fail to adapt to the computational complexity of input samples in a dynamic manner to minimize energy consumption. In this research, we propose DANCE, a fine-grained, input-aware, dynamic pruning framework for 3D CNNs to maximize power efficiency with negligible to zero impact on performance. In the proposed two-step approach, the first step is called activation variability amplification (AVA), and the 3D CNN model is retrained to increase the variance of the magnitude of neuron activations across the network in this step, facilitating pruning decisions across diverse CNN input scenarios. In the second step, called adaptive activation pruning (AAP), a lightweight activation controller network is trained to dynamically prune frames, channels, and features of 3D convolutional layers of the network (different for each layer), based on statistics of the outputs of the first layer of the network. Our method achieves substantial savings in multiply-accumulate (MAC) operations and memory accesses by introducing sparsity within convolutional layers. Hardware validation on the NVIDIA Jetson Nano GPU and the Qualcomm Snapdragon 8 Gen 1 platform demonstrates respective speedups of 1.37X and 2.22X, achieving up to 1.47X higher energy efficiency compared to the state of the art.