Layer-wise Weight Selection for Power-Efficient Neural Network Acceleration

TL;DR

This paper introduces a layer-wise, energy-aware compression method for systolic array CNN accelerators that significantly reduces energy consumption while maintaining high accuracy, by explicitly modeling layer-specific energy characteristics.

Contribution

It presents a novel layer-wise energy model and an energy-optimized weight selection algorithm integrated into quantization aware training for improved power efficiency.

Findings

Achieves up to 58.6% energy reduction

Maintains 2-3% accuracy drop

Outperforms existing power-aware methods

Abstract

Systolic array accelerators execute CNNs with energy dominated by the switching activity of multiply accumulate (MAC) units. Although prior work exploits weight dependent MAC power for compression, existing methods often use global activation models, coarse energy proxies, or layer-agnostic policies, which limits their effectiveness on real hardware. We propose an energy aware, layer-wise compression framework that explicitly leverages MAC and layer level energy characteristics. First, we build a layer-aware MAC energy model that combines per-layer activation statistics with an MSB-Hamming distance grouping of 22-bit partial sum transitions, and integrate it with a tile-level systolic mapping to estimate convolution-layer energy. On top of this model, we introduce an energy accuracy co-optimized weight selection algorithm within quantization aware training and an energy-prioritized layer-wise schedule that compresses high energy layers more aggressively under a global accuracy constraint. Experiments on different CNN models demonstrate up to 58.6\% energy reduction with 2-3\% accuracy drop, outperforming a state-of-the-art power-aware baseline.