RV-FuseNet: Range View Based Fusion of Time-Series LiDAR Data for Joint 3D Object Detection and Motion Forecasting

TL;DR

RV-FuseNet introduces a novel range view-based fusion method for LiDAR data that enhances joint 3D object detection and motion forecasting in autonomous vehicles by preserving data resolution and minimizing information loss.

Contribution

The paper proposes Incremental Fusion, a new architecture for sequentially projecting LiDAR range views to improve temporal fusion and motion prediction accuracy.

Findings

Significantly improves motion forecasting performance.

Outperforms existing RV-based fusion methods on multiple datasets.

Preserves full sensor resolution by avoiding voxelization.

Abstract

Robust real-time detection and motion forecasting of traffic participants is necessary for autonomous vehicles to safely navigate urban environments. In this paper, we present RV-FuseNet, a novel end-to-end approach for joint detection and trajectory estimation directly from time-series LiDAR data. Instead of the widely used bird's eye view (BEV) representation, we utilize the native range view (RV) representation of LiDAR data. The RV preserves the full resolution of the sensor by avoiding the voxelization used in the BEV. Furthermore, RV can be processed efficiently due to its compactness. Previous approaches project time-series data to a common viewpoint for temporal fusion, and often this viewpoint is different from where it was captured. This is sufficient for BEV methods, but for RV methods, this can lead to loss of information and data distortion which has an adverse impact on performance. To address this challenge we propose a simple yet effective novel architecture, \textit{Incremental Fusion}, that minimizes the information loss by sequentially projecting each RV sweep into the viewpoint of the next sweep in time. We show that our approach significantly improves motion forecasting performance over the existing state-of-the-art. Furthermore, we demonstrate that our sequential fusion approach is superior to alternative RV based fusion methods on multiple datasets.