End-to-end codesign of Hessian-aware quantized neural networks for FPGAs and ASICs

TL;DR

This paper presents an end-to-end workflow for designing and deploying Hessian-aware quantized neural networks on FPGA and ASIC hardware, enabling real-time applications in scientific and industrial fields.

Contribution

It introduces a comprehensive, open-source pipeline combining Hessian-aware quantization, QONNX, and hls4ml for efficient hardware implementation of neural networks.

Findings

Successfully deployed a neural network for particle jet classification at CERN.

Achieved real-time processing at 40 MHz collision rate with optimized hardware.

Demonstrated accessibility of the workflow for nonexperts in hardware design.

Abstract

We develop an end-to-end workflow for the training and implementation of co-designed neural networks (NNs) for efficient field-programmable gate array (FPGA) and application-specific integrated circuit (ASIC) hardware. Our approach leverages Hessian-aware quantization (HAWQ) of NNs, the Quantized Open Neural Network Exchange (QONNX) intermediate representation, and the hls4ml tool flow for transpiling NNs into FPGA and ASIC firmware. This makes efficient NN implementations in hardware accessible to nonexperts, in a single open-sourced workflow that can be deployed for real-time machine learning applications in a wide range of scientific and industrial settings. We demonstrate the workflow in a particle physics application involving trigger decisions that must operate at the 40 MHz collision rate of the CERN Large Hadron Collider (LHC). Given the high collision rate, all data processing must be implemented on custom ASIC and FPGA hardware within a strict area and latency. Based on these constraints, we implement an optimized mixed-precision NN classifier for high-momentum particle jets in simulated LHC proton-proton collisions.