DeepCut: Unsupervised Segmentation using Graph Neural Networks Clustering

TL;DR

This paper introduces DeepCut, a graph neural network-based method for unsupervised image segmentation that directly optimizes clustering objectives, enabling semantic segmentation without post-processing and outperforming existing methods.

Contribution

The study proposes a novel GNN approach that incorporates raw features and affinity information to perform clustering directly, eliminating the need for traditional post-processing steps.

Findings

Outperforms state-of-the-art on multiple benchmarks

Enables k-less clustering with correlation-clustering objective

Achieves part semantic segmentation without additional steps

Abstract

Image segmentation is a fundamental task in computer vision. Data annotation for training supervised methods can be labor-intensive, motivating unsupervised methods. Current approaches often rely on extracting deep features from pre-trained networks to construct a graph, and classical clustering methods like k-means and normalized-cuts are then applied as a post-processing step. However, this approach reduces the high-dimensional information encoded in the features to pair-wise scalar affinities. To address this limitation, this study introduces a lightweight Graph Neural Network (GNN) to replace classical clustering methods while optimizing for the same clustering objective function. Unlike existing methods, our GNN takes both the pair-wise affinities between local image features and the raw features as input. This direct connection between the raw features and the clustering objective enables us to implicitly perform classification of the clusters between different graphs, resulting in part semantic segmentation without the need for additional post-processing steps. We demonstrate how classical clustering objectives can be formulated as self-supervised loss functions for training an image segmentation GNN. Furthermore, we employ the Correlation-Clustering (CC) objective to perform clustering without defining the number of clusters, allowing for k-less clustering. We apply the proposed method for object localization, segmentation, and semantic part segmentation tasks, surpassing state-of-the-art performance on multiple benchmarks.