LiVisSfM: Accurate and Robust Structure-from-Motion with LiDAR and Visual Cues

TL;DR

LiVisSfM introduces a novel LiDAR-visual SfM pipeline that achieves high accuracy and robustness in 3D reconstruction without relying on IMU data, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a new LiDAR-visual SfM method with LiDAR frame registration to voxel maps, bundle adjustment, and loop closure, eliminating the need for IMU integration.

Findings

Outperforms state-of-the-art LIO and LIVO methods in accuracy and robustness.

Provides dense point cloud reconstructions on public and self-captured datasets.

Demonstrates effective LiDAR pose recovery and map updating strategies.

Abstract

This paper presents an accurate and robust Structure-from-Motion (SfM) pipeline named LiVisSfM, which is an SfM-based reconstruction system that fully combines LiDAR and visual cues. Unlike most existing LiDAR-inertial odometry (LIO) and LiDAR-inertial-visual odometry (LIVO) methods relying heavily on LiDAR registration coupled with Inertial Measurement Unit (IMU), we propose a LiDAR-visual SfM method which innovatively carries out LiDAR frame registration to LiDAR voxel map in a Point-to-Gaussian residual metrics, combined with a LiDAR-visual BA and explicit loop closure in a bundle optimization way to achieve accurate and robust LiDAR pose estimation without dependence on IMU incorporation. Besides, we propose an incremental voxel updating strategy for efficient voxel map updating during the process of LiDAR frame registration and LiDAR-visual BA optimization. Experiments demonstrate the superior effectiveness of our LiVisSfM framework over state-of-the-art LIO and LIVO works on more accurate and robust LiDAR pose recovery and dense point cloud reconstruction of both public KITTI benchmark and a variety of self-captured dataset.