Implementation and Analysis of Thermometer Encoding in DWN FPGA Accelerators

TL;DR

This paper evaluates the hardware costs of thermometer encoding in differential weightless neural network (DWN) FPGA accelerators, revealing significant resource increases and emphasizing the importance of encoding-aware design.

Contribution

It introduces a DWN hardware generator that explicitly models thermometer encoding, providing insights into its impact on FPGA resource usage.

Findings

Thermometer encoding can increase LUT usage by up to 3.20 times.

Encoding costs dominate in small DWN networks.

Highlights the need for encoding-aware hardware design in DWN accelerators.

Abstract

Fully parallel neural network accelerators on field-programmable gate arrays (FPGAs) offer high throughput for latency-critical applications but face hardware resource constraints. Weightless neural networks (WNNs) efficiently replace arithmetic with logic-based inference. Differential weightless neural networks (DWN) further optimize resource usage by learning connections between encoders and LUT layers via gradient-based training. However, DWNs rely on thermometer encoding, and the associated hardware cost has not been fully evaluated. We present a DWN hardware generator that includes thermometer encoding explicitly. Experiments on the Jet Substructure Classification (JSC) task show that encoding can increase LUT usage by up to 3.20$\times$, dominating costs in small networks and highlighting the need for encoding-aware hardware design in DWN accelerators.