rule4ml: An Open-Source Tool for Resource Utilization and Latency Estimation for ML Models on FPGA

TL;DR

This paper presents rule4ml, an open-source tool that predicts FPGA resource utilization and latency of neural networks before synthesis, significantly speeding up FPGA-based ML model development.

Contribution

It introduces trained regression models for pre-synthesis prediction of FPGA resource usage and latency, reducing development time for FPGA-implemented neural networks.

Findings

High prediction accuracy with R2 scores between 0.8 and 0.98

Predictions for BRAM, DSP, FF, LUT, and clock cycles are reliable

Accelerates FPGA neural network deployment process

Abstract

Implementing Machine Learning (ML) models on Field-Programmable Gate Arrays (FPGAs) is becoming increasingly popular across various domains as a low-latency and low-power solution that helps manage large data rates generated by continuously improving detectors. However, developing ML models for FPGAs is time-consuming, as optimization requires synthesis to evaluate FPGA area and latency, making the process slow and repetitive. This paper introduces a novel method to predict the resource utilization and inference latency of Neural Networks (NNs) before their synthesis and implementation on FPGA. We leverage HLS4ML, a tool-flow that helps translate NNs into high-level synthesis (HLS) code, to synthesize a diverse dataset of NN architectures and train resource utilization and inference latency predictors. While HLS4ML requires full synthesis to obtain resource and latency insights, our method uses trained regression models for immediate pre-synthesis predictions. The prediction models estimate the usage of Block RAM (BRAM), Digital Signal Processors (DSP), Flip-Flops (FF), and Look-Up Tables (LUT), as well as the inference clock cycles. The predictors were evaluated on both synthetic and existing benchmark architectures and demonstrated high accuracy with R2 scores ranging between 0.8 and 0.98 on the validation set and sMAPE values between 10% and 30%. Overall, our approach provides valuable preliminary insights, enabling users to quickly assess the feasibility and efficiency of NNs on FPGAs, accelerating the development and deployment processes. The open-source repository can be found at https://github.com/IMPETUS-UdeS/rule4ml, while the datasets are publicly available at https://borealisdata.ca/dataverse/rule4ml.