PointNet on FPGA for Real-Time LiDAR Point Cloud Processing

TL;DR

This paper develops an FPGA-based platform implementing PointNet for real-time LiDAR point cloud processing in autonomous vehicles, achieving high performance and low latency on a Xilinx Zynq UltraScale+ MPSoC.

Contribution

It introduces a hardware-accelerated FPGA implementation of PointNet, optimized for real-time processing of LiDAR data in autonomous vehicle applications.

Findings

Achieves 182.1 GOPS for classification

Supports up to 4096 points per frame

Processing times meet real-time requirements

Abstract

LiDAR sensors have been widely used in many autonomous vehicle modalities, such as perception, mapping, and localization. This paper presents an FPGA-based deep learning platform for real-time point cloud processing targeted on autonomous vehicles. The software driver for the Velodyne LiDAR sensor is modified and moved into the on-chip processor system, while the programmable logic is designed as a customized hardware accelerator. As the state-of-art deep learning algorithm for point cloud processing, PointNet is successfully implemented on the proposed FPGA platform. Targeted on a Xilinx Zynq UltraScale+ MPSoC ZCU104 development board, the FPGA implementations of PointNet achieve the computing performance of 182.1 GOPS and 280.0 GOPS for classification and segmentation respectively. The proposed design can support an input up to 4096 points per frame. The processing time is 19.8 ms for classification and 34.6 ms for segmentation, which meets the real-time requirement for most of the existing LiDAR sensors.