Improving the Performance of a NoC-based CNN Accelerator with Gather Support

TL;DR

This paper proposes using gather packets in mesh-based NoCs with output stationary systolic arrays to efficiently handle many-to-one data traffic in CNN accelerators, improving latency and power consumption.

Contribution

It introduces a gather packet mechanism for NoC-based CNN accelerators to optimize many-to-one traffic handling, enhancing performance over traditional unicast methods.

Findings

Reduced latency in CNN data transfer

Lower power consumption in NoC communication

Improved efficiency with gather packets on AlexNet and VGG-16

Abstract

The increasing application of deep learning technology drives the need for an efficient parallel computing architecture for Convolutional Neural Networks (CNNs). A significant challenge faced when designing a many-core CNN accelerator is to handle the data movement between the processing elements. The CNN workload introduces many-to-one traffic in addition to one-to-one and one-to-many traffic. As the de-facto standard for on-chip communication, Network-on-Chip (NoC) can support various unicast and multicast traffic. For many-to-one traffic, repetitive unicast is employed which is not an efficient way. In this paper, we propose to use the gather packet on mesh-based NoCs employing output stationary systolic array in support of many-to-one traffic. The gather packet will collect the data from the intermediate nodes eventually leading to the destination efficiently. This method is evaluated using the traffic traces generated from the convolution layer of AlexNet and VGG-16 with improvement in the latency and power than the repetitive unicast method.