Addressing Data Misalignment in Image-LiDAR Fusion on Point Cloud Segmentation

TL;DR

This paper investigates the problem of data misalignment in camera and LiDAR fusion for point cloud segmentation, proposing solutions to improve alignment accuracy and segmentation performance in autonomous driving perception systems.

Contribution

It introduces methods to address data misalignment issues in multi-sensor fusion, enhancing the accuracy of point cloud segmentation in autonomous driving.

Findings

Misalignment of LiDAR points on images affects segmentation accuracy

Proposed solutions improve data alignment and segmentation performance

Focus on nuScenes dataset and 2DPASS model

Abstract

With the advent of advanced multi-sensor fusion models, there has been a notable enhancement in the performance of perception tasks within in terms of autonomous driving. Despite these advancements, the challenges persist, particularly in the fusion of data from cameras and LiDAR sensors. A critial concern is the accurate alignment of data from these disparate sensors. Our observations indicate that the projected positions of LiDAR points often misalign on the corresponding image. Furthermore, fusion models appear to struggle in accurately segmenting these misaligned points. In this paper, we would like to address this problem carefully, with a specific focus on the nuScenes dataset and the SOTA of fusion models 2DPASS, and providing the possible solutions or potential improvements.