A Systematic Approach to Blocking Convolutional Neural Networks

TL;DR

This paper develops an analytical model for blocking CNN computations to optimize memory locality, leading to significant energy efficiency improvements and reduced memory accesses in hardware and CPU implementations.

Contribution

It introduces a systematic, analytical approach to determine optimal blocking strategies for CNNs, surpassing heuristic methods and improving efficiency.

Findings

Energy efficiency improved by up to 10x in hardware implementations.

Memory accesses reduced by up to 90% in optimized CPU programs.

Automated derivation of blocking strategies outperforms traditional heuristics.

Abstract

Convolutional Neural Networks (CNNs) are the state of the art solution for many computer vision problems, and many researchers have explored optimized implementations. Most implementations heuristically block the computation to deal with the large data sizes and high data reuse of CNNs. This paper explores how to block CNN computations for memory locality by creating an analytical model for CNN-like loop nests. Using this model we automatically derive optimized blockings for common networks that improve the energy efficiency of custom hardware implementations by up to an order of magnitude. Compared to traditional CNN CPU implementations based on highly-tuned, hand-optimized BLAS libraries,our x86 programs implementing the optimal blocking reduce the number of memory accesses by up to 90%.