NullaNet Tiny: Ultra-low-latency DNN Inference Through Fixed-function Combinational Logic

TL;DR

NullaNet Tiny introduces an FPGA-based neural network accelerator that replaces complex operations with Boolean logic, achieving ultra-low latency and high resource efficiency for applications demanding sub-microsecond response times.

Contribution

The paper presents a novel across-the-stack framework that maps neural network operations to FPGA LUTs, significantly reducing latency and resource usage compared to existing FPGA accelerators.

Findings

Achieves 2.36× lower latency than LogicNets.

Uses 24.42× fewer LUTs than LogicNets.

Maintains similar classification accuracy.

Abstract

While there is a large body of research on efficient processing of deep neural networks (DNNs), ultra-low-latency realization of these models for applications with stringent, sub-microsecond latency requirements continues to be an unresolved, challenging problem. Field-programmable gate array (FPGA)-based DNN accelerators are gaining traction as a serious contender to replace graphics processing unit/central processing unit-based platforms considering their performance, flexibility, and energy efficiency. This paper presents NullaNet Tiny, an across-the-stack design and optimization framework for constructing resource and energy-efficient, ultra-low-latency FPGA-based neural network accelerators. The key idea is to replace expensive operations required to compute various filter/neuron functions in a DNN with Boolean logic expressions that are mapped to the native look-up tables (LUTs) of the FPGA device (examples of such operations are multiply-and-accumulate and batch normalization). At about the same level of classification accuracy, compared to Xilinx's LogicNets, our design achieves 2.36$\times$ lower latency and 24.42$\times$ lower LUT utilization.