Cnvlutin2: Ineffectual-Activation-and-Weight-Free Deep Neural Network Computing

TL;DR

Cnvlutin2 introduces advanced techniques to efficiently skip ineffectual activations and weights in deep neural network accelerators, reducing memory usage and energy consumption through novel encoding, memory access, and detection methods.

Contribution

The paper presents new encoding schemes, memory access strategies, and an extended architecture for skipping ineffectual activations and weights in DNN accelerators.

Findings

Reduced memory footprint for activations

Lower energy consumption in DNN processing

Effective skipping of ineffectual weights and activations

Abstract

We discuss several modifications and extensions over the previous proposed Cnvlutin (CNV) accelerator for convolutional and fully-connected layers of Deep Learning Network. We first describe different encodings of the activations that are deemed ineffectual. The encodings have different memory overhead and energy characteristics. We propose using a level of indirection when accessing activations from memory to reduce their memory footprint by storing only the effectual activations. We also present a modified organization that detects the activations that are deemed as ineffectual while fetching them from memory. This is different than the original design that instead detected them at the output of the preceding layer. Finally, we present an extended CNV that can also skip ineffectual weights.