Energy-Efficient Inference Accelerator for Memory-Augmented Neural Networks on an FPGA

TL;DR

This paper presents an FPGA-based accelerator for memory-augmented neural networks, achieving significantly higher energy efficiency than GPUs, optimized for natural language question-answering tasks.

Contribution

The authors develop a novel FPGA accelerator tailored for MANNs, incorporating inference thresholding to enhance energy efficiency and inference speed on mobile-like hardware.

Findings

Energy efficiency 125x higher than GPU on bAbI data

Inference thresholding improves efficiency further

Effective FPGA implementation for MANNs on natural language tasks

Abstract

Memory-augmented neural networks (MANNs) are designed for question-answering tasks. It is difficult to run a MANN effectively on accelerators designed for other neural networks (NNs), in particular on mobile devices, because MANNs require recurrent data paths and various types of operations related to external memory access. We implement an accelerator for MANNs on a field-programmable gate array (FPGA) based on a data flow architecture. Inference times are also reduced by inference thresholding, which is a data-based maximum inner-product search specialized for natural language tasks. Measurements on the bAbI data show that the energy efficiency of the accelerator (FLOPS/kJ) was higher than that of an NVIDIA TITAN V GPU by a factor of about 125, increasing to 140 with inference thresholding