Unsupervised Learning of Image Segmentation Based on Differentiable Feature Clustering

TL;DR

This paper introduces a novel unsupervised image segmentation method using a differentiable clustering approach within CNNs, incorporating spatial continuity and user-guided extensions, validated on benchmark datasets.

Contribution

It presents a new end-to-end differentiable clustering network, a spatial continuity loss, and extensions for scribble-based and unseen image segmentation, advancing unsupervised segmentation techniques.

Findings

Effective on benchmark datasets

Outperforms existing scribble-based methods

Enables segmentation without re-training for new images

Abstract

The usage of convolutional neural networks (CNNs) for unsupervised image segmentation was investigated in this study. In the proposed approach, label prediction and network parameter learning are alternately iterated to meet the following criteria: (a) pixels of similar features should be assigned the same label, (b) spatially continuous pixels should be assigned the same label, and (c) the number of unique labels should be large. Although these criteria are incompatible, the proposed approach minimizes the combination of similarity loss and spatial continuity loss to find a plausible solution of label assignment that balances the aforementioned criteria well. The contributions of this study are four-fold. First, we propose a novel end-to-end network of unsupervised image segmentation that consists of normalization and an argmax function for differentiable clustering. Second, we introduce a spatial continuity loss function that mitigates the limitations of fixed segment boundaries possessed by previous work. Third, we present an extension of the proposed method for segmentation with scribbles as user input, which showed better accuracy than existing methods while maintaining efficiency. Finally, we introduce another extension of the proposed method: unseen image segmentation by using networks pre-trained with a few reference images without re-training the networks. The effectiveness of the proposed approach was examined on several benchmark datasets of image segmentation.