StixelNExT: Toward Monocular Low-Weight Perception for Object Segmentation and Free Space Detection

TL;DR

This paper introduces a monocular perception system that learns from LiDAR data during training to perform object segmentation and free space detection without manual labels, enabling lightweight and adaptable perception.

Contribution

It presents a novel training approach that uses LiDAR data for supervision, then removes it, allowing monocular perception with minimal data and no manual labeling.

Findings

Effective object segmentation from monocular images

Accurate free space detection demonstrated

Model trained with LiDAR supervision and deployed monocularly

Abstract

In this work, we present a novel approach for general object segmentation from a monocular image, eliminating the need for manually labeled training data and enabling rapid, straightforward training and adaptation with minimal data. Our model initially learns from LiDAR during the training process, which is subsequently removed from the system, allowing it to function solely on monocular imagery. This study leverages the concept of the Stixel-World to recognize a medium level representation of its surroundings. Our network directly predicts a 2D multi-layer Stixel-World and is capable of recognizing and locating multiple, superimposed objects within an image. Due to the scarcity of comparable works, we have divided the capabilities into modules and present a free space detection in our experiments section. Furthermore, we introduce an improved method for generating Stixels from LiDAR data, which we use as ground truth for our network.