Real-Time Graph-based Point Cloud Networks on FPGAs via Stall-Free Deep Pipelining

TL;DR

This paper presents a FPGA-based deep pipelined architecture for graph-based point cloud networks, achieving high throughput and low latency suitable for real-time physics experiments.

Contribution

It introduces a novel FPGA implementation with specialized processing elements for graph operations, enabling real-time processing of dynamic, sparse point clouds.

Findings

Up to 5.25x throughput speedup over GPU baseline

Latencies maintained below 10 microseconds

Supports real-time constraints in high-energy physics detectors

Abstract

Graph-based Point Cloud Networks (PCNs) are powerful tools for processing sparse sensor data with irregular geometries, as found in high-energy physics detectors. However, deploying models in such environments remains challenging due to stringent real-time requirements for both latency, and throughput. In this work, we present a deeply pipelined dataflow architecture for executing graph-based PCNs on FPGAs. Our method supports efficient processing of dynamic, sparse point clouds while meeting hard real-time constraints. We introduce specialized processing elements for core graph operations, such as GraVNet convolution and condensation point clustering, and demonstrate our design on the AMD Versal VCK190. Compared to a GPU baseline, our FPGA implementation achieves up to 5.25x speedup in throughput while maintaining latencies below 10 {\mu}s, satisfying the demands of real-time trigger systems in particle physics experiments. An open-source reference implementation is provided.