CUTIE: Beyond PetaOp/s/W Ternary DNN Inference Acceleration with Better-than-Binary Energy Efficiency

TL;DR

This paper introduces CUTIE, a digital hardware accelerator for ternary neural networks that achieves unprecedented energy efficiency and performance, surpassing binary neural network accelerators in energy savings and accuracy.

Contribution

The paper presents a fully digital, unrolled hardware architecture for ternary neural networks that significantly reduces energy consumption and switching activity, with an optimized training method for higher sparsity.

Findings

Achieves 3.1 POp/s/W energy efficiency

Reduces inference energy cost by up to 21x

Maintains or improves accuracy compared to state-of-the-art

Abstract

We present a 3.1 POp/s/W fully digital hardware accelerator for ternary neural networks. CUTIE, the Completely Unrolled Ternary Inference Engine, focuses on minimizing non-computational energy and switching activity so that dynamic power spent on storing (locally or globally) intermediate results is minimized. This is achieved by 1) a data path architecture completely unrolled in the feature map and filter dimensions to reduce switching activity by favoring silencing over iterative computation and maximizing data re-use, 2) targeting ternary neural networks which, in contrast to binary NNs, allow for sparse weights which reduce switching activity, and 3) introducing an optimized training method for higher sparsity of the filter weights, resulting in a further reduction of the switching activity. Compared with state-of-the-art accelerators, CUTIE achieves greater or equal accuracy while decreasing the overall core inference energy cost by a factor of 4.8x-21x.