Efficient Recurrent Neural Networks using Structured Matrices in FPGAs

TL;DR

This paper introduces a method using structured block-circulant matrices to efficiently implement RNNs on FPGAs, achieving significant energy efficiency improvements while maintaining accuracy.

Contribution

It proposes a novel weight matrix representation for RNNs using block-circulant matrices, enabling model compression and acceleration on FPGA with minimal accuracy loss.

Findings

Achieves up to 35.7× energy efficiency improvement over ESE.

Demonstrates effective RNN deployment on FPGA with negligible accuracy degradation.

Provides a framework for efficient RNN implementation using structured matrices.

Abstract

Recurrent Neural Networks (RNNs) are becoming increasingly important for time series-related applications which require efficient and real-time implementations. The recent pruning based work ESE suffers from degradation of performance/energy efficiency due to the irregular network structure after pruning. We propose block-circulant matrices for weight matrix representation in RNNs, thereby achieving simultaneous model compression and acceleration. We aim to implement RNNs in FPGA with highest performance and energy efficiency, with certain accuracy requirement (negligible accuracy degradation). Experimental results on actual FPGA deployments shows that the proposed framework achieves a maximum energy efficiency improvement of 35.7$\times$ compared with ESE.