Slanted Stixels: A way to represent steep streets

TL;DR

This paper introduces Slanted Stixels, a compact scene representation that models non-flat roads and slanted objects using a novel depth model, combined with semantic cues, enabling real-time processing and improved accuracy on non-flat terrains.

Contribution

It proposes a new depth model for Stixels to handle slanted surfaces, a novel over-segmentation strategy with FCN, and achieves real-time performance with enhanced accuracy on non-flat roads.

Findings

Maintains accuracy on flat road datasets.

Substantially improves accuracy on non-flat road datasets.

Achieves real-time computation capabilities.

Abstract

This work presents and evaluates a novel compact scene representation based on Stixels that infers geometric and semantic information. Our approach overcomes the previous rather restrictive geometric assumptions for Stixels by introducing a novel depth model to account for non-flat roads and slanted objects. Both semantic and depth cues are used jointly to infer the scene representation in a sound global energy minimization formulation. Furthermore, a novel approximation scheme is introduced in order to significantly reduce the computational complexity of the Stixel algorithm, and then achieve real-time computation capabilities. The idea is to first perform an over-segmentation of the image, discarding the unlikely Stixel cuts, and apply the algorithm only on the remaining Stixel cuts. This work presents a novel over-segmentation strategy based on a Fully Convolutional Network (FCN), which outperforms an approach based on using local extrema of the disparity map. We evaluate the proposed methods in terms of semantic and geometric accuracy as well as run-time on four publicly available benchmark datasets. Our approach maintains accuracy on flat road scene datasets while improving substantially on a novel non-flat road dataset.