Open Challenges in Deep Stereo: the Booster Dataset

TL;DR

This paper introduces the Booster Dataset, a high-resolution stereo dataset with challenging indoor scenes featuring transparent and specular surfaces, to evaluate and improve deep stereo networks.

Contribution

The paper presents a novel high-resolution stereo dataset with challenging features and a new acquisition pipeline using deep space-time stereo for accurate labeling.

Findings

State-of-the-art networks struggle with transparent and specular surfaces.

The dataset reveals limitations of current stereo methods.

Provides a benchmark for future stereo research.

Abstract

We present a novel high-resolution and challenging stereo dataset framing indoor scenes annotated with dense and accurate ground-truth disparities. Peculiar to our dataset is the presence of several specular and transparent surfaces, i.e. the main causes of failures for state-of-the-art stereo networks. Our acquisition pipeline leverages a novel deep space-time stereo framework which allows for easy and accurate labeling with sub-pixel precision. We release a total of 419 samples collected in 64 different scenes and annotated with dense ground-truth disparities. Each sample include a high-resolution pair (12 Mpx) as well as an unbalanced pair (Left: 12 Mpx, Right: 1.1 Mpx). Additionally, we provide manually annotated material segmentation masks and 15K unlabeled samples. We evaluate state-of-the-art deep networks based on our dataset, highlighting their limitations in addressing the open challenges in stereo and drawing hints for future research.