Real-Time LiDAR Point Cloud Densification for Low-Latency Spatial Data Transmission

TL;DR

This paper introduces a real-time LiDAR point cloud densification method that combines multiple data sources and neural filtering to produce dense, accurate 3D scenes at 30 fps, enabling low-latency immersive telepresence.

Contribution

It presents a novel high-speed densification approach integrating multi-modal data and neural filtering, achieving real-time dense 3D scene generation with high accuracy and minimal latency.

Findings

Produces dense depth maps at 30 fps in real time

Over 15x faster than recent depth completion methods

Dense point clouds with accurate geometry and no artifacts

Abstract

To realize low-latency spatial transmission system for immersive telepresence, there are two major problems: capturing dynamic 3D scene densely and processing them in real time. LiDAR sensors capture 3D in real time, but produce sparce point clouds. Therefore, this paper presents a high-speed LiDAR point cloud densification method to generate dense 3D scene with minimal latency, addressing the need for on-the-fly depth completion while maintaining real-time performance. Our approach combines multiple LiDAR inputs with high-resolution color images and applies a joint bilateral filtering strategy implemented through a convolutional neural network architecture. Experiments demonstrate that the proposed method produces dense depth maps at full HD resolution in real time (30 fps), which is over 15x faster than a recent training-based depth completion approach. The resulting dense point clouds exhibit accurate geometry without multiview inconsistencies or ghosting artifacts.