Learning Stixel-based Instance Segmentation

TL;DR

StixelPointNet introduces a fast, effective method for instance segmentation directly on Stixels by leveraging PointNet architecture, enabling new 3D deep learning applications in autonomous driving.

Contribution

The paper presents a novel approach that applies PointNet to Stixels for efficient instance segmentation, bridging the gap between sparse Stixel data and deep learning.

Findings

Achieves state-of-the-art Stixel-level segmentation performance

Significantly faster than pixel-based methods

Enables new 3D deep learning tasks with Stixels

Abstract

Stixels have been successfully applied to a wide range of vision tasks in autonomous driving, recently including instance segmentation. However, due to their sparse occurrence in the image, until now Stixels seldomly served as input for Deep Learning algorithms, restricting their utility for such approaches. In this work we present StixelPointNet, a novel method to perform fast instance segmentation directly on Stixels. By regarding the Stixel representation as unstructured data similar to point clouds, architectures like PointNet are able to learn features from Stixels. We use a bounding box detector to propose candidate instances, for which the relevant Stixels are extracted from the input image. On these Stixels, a PointNet models learns binary segmentations, which we then unify throughout the whole image in a final selection step. StixelPointNet achieves state-of-the-art performance on Stixel-level, is considerably faster than pixel-based segmentation methods, and shows that with our approach the Stixel domain can be introduced to many new 3D Deep Learning tasks.