Analyzing General-Purpose Deep-Learning Detection and Segmentation Models with Images from a Lidar as a Camera Sensor

TL;DR

This paper investigates the use of general-purpose deep learning detection and segmentation models on low-resolution lidar-derived images, demonstrating their potential to enhance perception in challenging environmental conditions.

Contribution

It is the first to analyze the application of vision-based deep learning models on lidar-generated images, expanding perception capabilities beyond traditional point cloud processing.

Findings

Deep learning models can effectively process lidar-derived images with proper preprocessing.

Lidar images enable perception in conditions where traditional vision sensors struggle.

The approach leverages mature visual perception models for lidar data analysis.

Abstract

Over the last decade, robotic perception algorithms have significantly benefited from the rapid advances in deep learning (DL). Indeed, a significant amount of the autonomy stack of different commercial and research platforms relies on DL for situational awareness, especially vision sensors. This work explores the potential of general-purpose DL perception algorithms, specifically detection and segmentation neural networks, for processing image-like outputs of advanced lidar sensors. Rather than processing the three-dimensional point cloud data, this is, to the best of our knowledge, the first work to focus on low-resolution images with 360\textdegree field of view obtained with lidar sensors by encoding either depth, reflectivity, or near-infrared light in the image pixels. We show that with adequate preprocessing, general-purpose DL models can process these images, opening the door to their usage in environmental conditions where vision sensors present inherent limitations. We provide both a qualitative and quantitative analysis of the performance of a variety of neural network architectures. We believe that using DL models built for visual cameras offers significant advantages due to the much wider availability and maturity compared to point cloud-based perception.