ViLiVO: Virtual LiDAR-Visual Odometry for an Autonomous Vehicle with a Multi-Camera System

TL;DR

This paper introduces ViLiVO, a multi-camera visual odometry system that synthesizes virtual LiDAR scans from fisheye camera images and fuses them with feature-based methods for robust pose estimation in autonomous vehicles.

Contribution

The paper presents a novel multi-camera VO system combining virtual LiDAR generation and feature matching, improving robustness and accuracy over traditional monocular systems.

Findings

Enhanced pose estimation accuracy in diverse environments

Robustness against challenging lighting and occlusion conditions

Effective integration of virtual LiDAR with feature-based VO

Abstract

In this paper, we present a multi-camera visual odometry (VO) system for an autonomous vehicle. Our system mainly consists of a virtual LiDAR and a pose tracker. We use a perspective transformation method to synthesize a surround-view image from undistorted fisheye camera images. With a semantic segmentation model, the free space can be extracted. The scans of the virtual LiDAR are generated by discretizing the contours of the free space. As for the pose tracker, we propose a visual odometry system fusing both the feature matching and the virtual LiDAR scan matching results. Only those feature points located in the free space area are utilized to ensure the 2D-2D matching for pose estimation. Furthermore, bundle adjustment (BA) is performed to minimize the feature points reprojection error and scan matching error. We apply our system to an autonomous vehicle equipped with four fisheye cameras. The testing scenarios include an outdoor parking lot as well as an indoor garage. Experimental results demonstrate that our system achieves a more robust and accurate performance comparing with a fisheye camera based monocular visual odometry system.