MVTec D2S: Densely Segmented Supermarket Dataset

TL;DR

The paper introduces D2S, a comprehensive and challenging dataset for instance-aware semantic segmentation in industrial settings, designed to evaluate and improve segmentation methods under real-world conditions.

Contribution

It presents a new large-scale dataset with high-resolution images, detailed annotations, and diverse scenarios for benchmarking segmentation algorithms in industrial applications.

Findings

Current methods show significant room for improvement on D2S.

The dataset highlights challenges like limited training data and high test diversity.

Robustness of segmentation methods varies across different lighting and background conditions.

Abstract

We introduce the Densely Segmented Supermarket (D2S) dataset, a novel benchmark for instance-aware semantic segmentation in an industrial domain. It contains 21,000 high-resolution images with pixel-wise labels of all object instances. The objects comprise groceries and everyday products from 60 categories. The benchmark is designed such that it resembles the real-world setting of an automatic checkout, inventory, or warehouse system. The training images only contain objects of a single class on a homogeneous background, while the validation and test sets are much more complex and diverse. To further benchmark the robustness of instance segmentation methods, the scenes are acquired with different lightings, rotations, and backgrounds. We ensure that there are no ambiguities in the labels and that every instance is labeled comprehensively. The annotations are pixel-precise and allow using crops of single instances for articial data augmentation. The dataset covers several challenges highly relevant in the field, such as a limited amount of training data and a high diversity in the test and validation sets. The evaluation of state-of-the-art object detection and instance segmentation methods on D2S reveals significant room for improvement.