Going Further With Winograd Convolutions: Tap-Wise Quantization for Efficient Inference on 4x4 Tile

TL;DR

This paper introduces a tap-wise quantization method and custom hardware to enable efficient, integer-only inference with larger Winograd tiles, significantly improving throughput and energy efficiency in neural network convolution operations.

Contribution

The paper presents a novel tap-wise quantization technique and specialized hardware units that unlock the potential of Winograd F4 convolutions for efficient neural network inference.

Findings

Achieves near FP32 accuracy with quantized Winograd F4.

Up to 1.85x energy efficiency gain on hardware.

Up to 1.83x end-to-end speed-up on vision networks.

Abstract

Most of today's computer vision pipelines are built around deep neural networks, where convolution operations require most of the generally high compute effort. The Winograd convolution algorithm computes convolutions with fewer MACs compared to the standard algorithm, reducing the operation count by a factor of 2.25x for 3x3 convolutions when using the version with 2x2-sized tiles $F_2$. Even though the gain is significant, the Winograd algorithm with larger tile sizes, i.e., $F_4$, offers even more potential in improving throughput and energy efficiency, as it reduces the required MACs by 4x. Unfortunately, the Winograd algorithm with larger tile sizes introduces numerical issues that prevent its use on integer domain-specific accelerators and higher computational overhead to transform input and output data between spatial and Winograd domains. To unlock the full potential of Winograd $F_4$, we propose a novel tap-wise quantization method that overcomes the numerical issues of using larger tiles, enabling integer-only inference. Moreover, we present custom hardware units that process the Winograd transformations in a power- and area-efficient way, and we show how to integrate such custom modules in an industrial-grade, programmable DSA. An extensive experimental evaluation on a large set of state-of-the-art computer vision benchmarks reveals that the tap-wise quantization algorithm makes the quantized Winograd $F_4$ network almost as accurate as the FP32 baseline. The Winograd-enhanced DSA achieves up to 1.85x gain in energy efficiency and up to 1.83x end-to-end speed-up for state-of-the-art segmentation and detection networks.