Large-scale Unsupervised Semantic Segmentation

TL;DR

This paper introduces the large-scale unsupervised semantic segmentation (LUSS) problem, creates a new ImageNet-S dataset for benchmarking, and proposes an effective method to advance research in unsupervised segmentation.

Contribution

It defines the LUSS problem, provides a large-scale benchmark dataset, and offers a simple method that performs well, facilitating progress in unsupervised semantic segmentation.

Findings

Created ImageNet-S dataset with 1.2 million images and 50k annotations.

Benchmarking of various supervised and unsupervised methods.

Identified challenges and future directions for LUSS.

Abstract

Empowered by large datasets, e.g., ImageNet, unsupervised learning on large-scale data has enabled significant advances for classification tasks. However, whether the large-scale unsupervised semantic segmentation can be achieved remains unknown. There are two major challenges: i) we need a large-scale benchmark for assessing algorithms; ii) we need to develop methods to simultaneously learn category and shape representation in an unsupervised manner. In this work, we propose a new problem of large-scale unsupervised semantic segmentation (LUSS) with a newly created benchmark dataset to help the research progress. Building on the ImageNet dataset, we propose the ImageNet-S dataset with 1.2 million training images and 50k high-quality semantic segmentation annotations for evaluation. Our benchmark has a high data diversity and a clear task objective. We also present a simple yet effective method that works surprisingly well for LUSS. In addition, we benchmark related un/weakly/fully supervised methods accordingly, identifying the challenges and possible directions of LUSS. The benchmark and source code is publicly available at https://github.com/LUSSeg.