The Cityscapes Dataset for Semantic Urban Scene Understanding

TL;DR

The paper introduces Cityscapes, a large-scale, richly annotated dataset of urban street scenes designed to advance semantic understanding in complex real-world environments, supporting the development of deep learning models.

Contribution

It presents a comprehensive dataset with high-quality annotations and an empirical analysis, surpassing previous datasets in size, diversity, and annotation detail for urban scene understanding.

Findings

Cityscapes dataset contains 5,000 high-quality annotated images.

200,000 images with coarse annotations enable weakly-supervised learning.

Performance evaluation of state-of-the-art methods highlights current challenges.

Abstract

Visual understanding of complex urban street scenes is an enabling factor for a wide range of applications. Object detection has benefited enormously from large-scale datasets, especially in the context of deep learning. For semantic urban scene understanding, however, no current dataset adequately captures the complexity of real-world urban scenes. To address this, we introduce Cityscapes, a benchmark suite and large-scale dataset to train and test approaches for pixel-level and instance-level semantic labeling. Cityscapes is comprised of a large, diverse set of stereo video sequences recorded in streets from 50 different cities. 5000 of these images have high quality pixel-level annotations; 20000 additional images have coarse annotations to enable methods that leverage large volumes of weakly-labeled data. Crucially, our effort exceeds previous attempts in terms of dataset size, annotation richness, scene variability, and complexity. Our accompanying empirical study provides an in-depth analysis of the dataset characteristics, as well as a performance evaluation of several state-of-the-art approaches based on our benchmark.