# Winograd Convolution for DNNs: Beyond linear polynomials

## TL;DR

This paper explores a broader set of Winograd algorithms for DNNs, demonstrating significant improvements in floating point accuracy and efficiency across various formats, surpassing traditional methods.

## Contribution

It introduces and evaluates a wider range of Winograd algorithms for DNNs, showing they can enhance accuracy and reduce computations compared to existing approaches.

## Key findings

- Up to 6.5 times better accuracy in fp16 for image recognition.
- Fewer innermost loop multiplications in bf16 without accuracy loss.
- Significant accuracy and efficiency improvements over traditional Winograd algorithms.

## Abstract

Winograd convolution is widely used in deep neural networks (DNNs). Existing work for DNNs considers only the subset Winograd algorithms that are equivalent to Toom-Cook convolution. We investigate a wider range of Winograd algorithms for DNNs and show that these additional algorithms can significantly improve floating point (FP) accuracy in many cases. We present results for three FP formats: fp32, fp16 and bf16 (a truncated form of fp32) using 2000 inputs from the ImageNet dataset. We found that in fp16 this approach gives us up to 6.5 times better image recognition accuracy in one important case while maintaining the same number of elementwise multiplication operations in the innermost loop. In bf16 the convolution can be computed using 5% fewer innermost loop multiplications than with currently used Winograd algorithms while keeping the accuracy of image recognition the same as for direct convolution method.