PLIN: A Network for Pseudo-LiDAR Point Cloud Interpolation

TL;DR

PLIN is a novel deep learning framework that enhances LiDAR point cloud frequency by generating high-quality intermediate 3D point clouds, improving multi-sensor data synchronization in autonomous systems.

Contribution

It introduces the first deep learning approach for Pseudo-LiDAR point cloud interpolation, combining coarse and refined stages for accurate high-frequency 3D data generation.

Findings

Outperforms traditional interpolation methods on KITTI dataset

Achieves higher accuracy in intermediate point cloud generation

Demonstrates potential for improved autonomous navigation systems

Abstract

LiDAR sensors can provide dependable 3D spatial information at a low frequency (around 10Hz) and have been widely applied in the field of autonomous driving and UAV. However, the camera with a higher frequency (around 20Hz) has to be decreased so as to match with LiDAR in a multi-sensor system. In this paper, we propose a novel Pseudo-LiDAR interpolation network (PLIN) to increase the frequency of LiDAR sensors. PLIN can generate temporally and spatially high-quality point cloud sequences to match the high frequency of cameras. To achieve this goal, we design a coarse interpolation stage guided by consecutive sparse depth maps and motion relationship. We also propose a refined interpolation stage guided by the realistic scene. Using this coarse-to-fine cascade structure, our method can progressively perceive multi-modal information and generate accurate intermediate point clouds. To the best of our knowledge, this is the first deep framework for Pseudo-LiDAR point cloud interpolation, which shows appealing applications in navigation systems equipped with LiDAR and cameras. Experimental results demonstrate that PLIN achieves promising performance on the KITTI dataset, significantly outperforming the traditional interpolation method and the state-of-the-art video interpolation technique.