Approximate FPGA-based LSTMs under Computation Time Constraints

TL;DR

This paper presents an FPGA-based LSTM architecture with approximation techniques like low-rank compression and pruning, enabling high-performance, time-constrained AI applications with significantly reduced computation time and improved accuracy.

Contribution

It introduces an end-to-end framework combining approximation methods and FPGA architecture optimization for efficient LSTM deployment under strict time constraints.

Findings

Achieved up to 6.5x reduction in computation time for the same accuracy.

Attained an average of 25x higher accuracy within the same time budget.

Demonstrated effectiveness on a real-life image captioning task.

Abstract

Recurrent Neural Networks and in particular Long Short-Term Memory (LSTM) networks have demonstrated state-of-the-art accuracy in several emerging Artificial Intelligence tasks. However, the models are becoming increasingly demanding in terms of computational and memory load. Emerging latency-sensitive applications including mobile robots and autonomous vehicles often operate under stringent computation time constraints. In this paper, we address the challenge of deploying computationally demanding LSTMs at a constrained time budget by introducing an approximate computing scheme that combines iterative low-rank compression and pruning, along with a novel FPGA-based LSTM architecture. Combined in an end-to-end framework, the approximation method's parameters are optimised and the architecture is configured to address the problem of high-performance LSTM execution in time-constrained applications. Quantitative evaluation on a real-life image captioning application indicates that the proposed methods required up to 6.5x less time to achieve the same application-level accuracy compared to a baseline method, while achieving an average of 25x higher accuracy under the same computation time constraints.