FPPN: Future Pseudo-LiDAR Frame Prediction for Autonomous Driving

TL;DR

This paper introduces a novel neural network that predicts future dense pseudo-LiDAR frames from sparse LiDAR and RGB data, enhancing 3D perception for autonomous driving by improving spatial and temporal density.

Contribution

It is the first to propose a future pseudo-LiDAR frame prediction network that fuses dynamic and static information to generate dense 3D point clouds from sparse inputs.

Findings

Outperforms existing methods on KITTI benchmark.

Effectively predicts future dense depth maps.

Improves 3D point cloud density and accuracy.

Abstract

LiDAR sensors are widely used in autonomous driving due to the reliable 3D spatial information. However, the data of LiDAR is sparse and the frequency of LiDAR is lower than that of cameras. To generate denser point clouds spatially and temporally, we propose the first future pseudo-LiDAR frame prediction network. Given the consecutive sparse depth maps and RGB images, we first predict a future dense depth map based on dynamic motion information coarsely. To eliminate the errors of optical flow estimation, an inter-frame aggregation module is proposed to fuse the warped depth maps with adaptive weights. Then, we refine the predicted dense depth map using static contextual information. The future pseudo-LiDAR frame can be obtained by converting the predicted dense depth map into corresponding 3D point clouds. Experimental results show that our method outperforms the existing solutions on the popular KITTI benchmark.