Unsupervised Image Segmentation by Mutual Information Maximization and Adversarial Regularization

TL;DR

This paper introduces InMARS, a fully unsupervised semantic segmentation method that leverages mutual information maximization and adversarial regularization to cluster superpixels into meaningful classes without labeled data.

Contribution

The paper presents a novel unsupervised segmentation approach combining mutual information maximization with adversarial regularization, inspired by human perception and capable of handling photometrical and geometrical variations.

Findings

Achieves state-of-the-art results on COCO-Stuff dataset

Outperforms existing unsupervised segmentation methods

Demonstrates robustness to noise and spatial perturbations

Abstract

Semantic segmentation is one of the basic, yet essential scene understanding tasks for an autonomous agent. The recent developments in supervised machine learning and neural networks have enjoyed great success in enhancing the performance of the state-of-the-art techniques for this task. However, their superior performance is highly reliant on the availability of a large-scale annotated dataset. In this paper, we propose a novel fully unsupervised semantic segmentation method, the so-called Information Maximization and Adversarial Regularization Segmentation (InMARS). Inspired by human perception which parses a scene into perceptual groups, rather than analyzing each pixel individually, our proposed approach first partitions an input image into meaningful regions (also known as superpixels). Next, it utilizes Mutual-Information-Maximization followed by an adversarial training strategy to cluster these regions into semantically meaningful classes. To customize an adversarial training scheme for the problem, we incorporate adversarial pixel noise along with spatial perturbations to impose photometrical and geometrical invariance on the deep neural network. Our experiments demonstrate that our method achieves the state-of-the-art performance on two commonly used unsupervised semantic segmentation datasets, COCO-Stuff, and Potsdam.