Patch-Based Deep Unsupervised Image Segmentation using Graph Cuts

TL;DR

This paper introduces a novel patch-based unsupervised image segmentation method that combines deep learning with classical graph cuts, achieving state-of-the-art results without supervision.

Contribution

It presents a new unsupervised segmentation approach that integrates deep feature extraction, classical graph cuts, and vision transformer features for improved accuracy.

Findings

Achieves state-of-the-art unsupervised segmentation performance.

Effectively leverages deep clustering and classical graph-based methods.

Demonstrates strong results on real image datasets.

Abstract

Unsupervised image segmentation aims at grouping different semantic patterns in an image without the use of human annotation. Similarly, image clustering searches for groupings of images based on their semantic content without supervision. Classically, both problems have captivated researchers as they drew from sound mathematical concepts to produce concrete applications. With the emergence of deep learning, the scientific community turned its attention to complex neural network-based solvers that achieved impressive results in those domains but rarely leveraged the advances made by classical methods. In this work, we propose a patch-based unsupervised image segmentation strategy that bridges advances in unsupervised feature extraction from deep clustering methods with the algorithmic help of classical graph-based methods. We show that a simple convolutional neural network, trained to classify image patches and iteratively regularized using graph cuts, naturally leads to a state-of-the-art fully-convolutional unsupervised pixel-level segmenter. Furthermore, we demonstrate that this is the ideal setting for leveraging the patch-level pairwise features generated by vision transformer models. Our results on real image data demonstrate the effectiveness of our proposed methodology.