Sidebar: Scratchpad Based Communication Between CPUs and Accelerators

TL;DR

This paper introduces Sidebar, a shared buffer architecture enabling flexible neural network accelerators that coordinate with host processors, maintaining high performance and energy efficiency while allowing dynamic computation of evolving functions.

Contribution

The paper proposes Sidebar, a low-latency shared buffer that facilitates flexible accelerator design with host-accelerator cooperation, improving over DMA-dependent approaches.

Findings

Sidebar achieves performance comparable to fixed-function accelerators.

Sidebar maintains energy efficiency similar to fixed-function designs.

Flexible accelerators with Sidebar outperform DMA-based flexible designs.

Abstract

Hardware accelerators for neural networks have shown great promise for both performance and power. These accelerators are at their most efficient when optimized for a fixed functionality. But this inflexibility limits the longevity of the hardware itself as the underlying neural network algorithms and structures undergo improvements and changes. We propose and evaluate a flexible design paradigm for accelerators with a close coordination with host processors. The relatively static matrix operations are implemented in specialized accelerators while fast-evolving functions, such as activations, are computed on the host processor. This architecture is enabled by a low latency shared buffer we call Sidebar. Sidebar memory is shared between the accelerator and host, exists outside of program address space and holds intermediate data only. We show that a generalised DMA dependent flexible accelerator design performs poorly in both perf and energy as compared to an equivalent fixed function accelerator. Sidebar based accelerator design achieves near identical performance and energy to equivalent fixed function accelerator while still providing all the flexibility of computing activations on the host processor.