CaRLi-V: Camera-RADAR-LiDAR Point-Wise 3D Velocity Estimation

TL;DR

CaRLi-V is a novel sensor fusion pipeline combining RADAR, LiDAR, and camera data to accurately estimate point-wise 3D velocities in dynamic environments, enhancing robotic interaction capabilities.

Contribution

It introduces a new RADAR representation called the velocity cube and a fusion method that outperforms existing scene flow techniques for 3D velocity estimation.

Findings

Achieves low velocity error metrics on a custom dataset.

Outperforms state-of-the-art scene flow methods.

Provides an open-source ROS2 package for practical use.

Abstract

Accurate point-wise velocity estimation in 3D is crucial for robot interaction with non-rigid dynamic agents, enabling robust performance in path planning, collision avoidance, and object manipulation in dynamic environments. To this end, this paper proposes a novel RADAR, LiDAR, and camera fusion pipeline for point-wise 3D velocity estimation named CaRLi-V. This pipeline leverages raw RADAR measurements to create a novel RADAR representation, the velocity cube, which densely encodes RADAR radial velocities. By combining the velocity cube for radial velocity extraction, optical flow for tangential velocity estimation, and LiDAR for point-wise range measurements through a closed-form solution, our approach can produce 3D velocity estimates for a dense array of points. Developed as an open-source ROS2 package, CaRLi-V has been field-tested on a custom dataset and achieves low velocity error metrics relative to ground truth while outperforming state-of-the-art scene flow methods.