Interleaver Design for Deep Neural Networks

TL;DR

This paper introduces a new class of interleavers for deep neural networks that enable structured sparsity, reducing memory and computation needs while improving performance and hardware efficiency.

Contribution

It presents a novel interleaver design algorithm with mathematical proofs, enhancing neural network efficiency and hardware implementation.

Findings

Interleavers reduce memory and computational requirements.

Interleavers improve network performance through optimized spread and dispersion.

Design algorithm guarantees clash-free memory access and hardware-friendly complexity.

Abstract

We propose a class of interleavers for a novel deep neural network (DNN) architecture that uses algorithmically pre-determined, structured sparsity to significantly lower memory and computational requirements, and speed up training. The interleavers guarantee clash-free memory accesses to eliminate idle operational cycles, optimize spread and dispersion to improve network performance, and are designed to ease the complexity of memory address computations in hardware. We present a design algorithm with mathematical proofs for these properties. We also explore interleaver variations and analyze the behavior of neural networks as a function of interleaver metrics.