Bootstrap Deep Spectral Clustering with Optimal Transport

TL;DR

This paper introduces BootSC, a deep spectral clustering model that jointly learns affinity construction, spectral embedding, and clustering in an end-to-end manner, significantly improving clustering performance with optimal transport supervision.

Contribution

The paper proposes a novel deep spectral clustering framework that integrates all stages into a single network and employs optimal transport for supervision, enhancing representation and clustering accuracy.

Findings

Achieves state-of-the-art clustering performance.

Improves NMI by 16% on ImageNet-Dogs dataset.

Introduces a semantically-consistent orthogonal re-parameterization technique.

Abstract

Spectral clustering is a leading clustering method. Two of its major shortcomings are the disjoint optimization process and the limited representation capacity. To address these issues, we propose a deep spectral clustering model (named BootSC), which jointly learns all stages of spectral clustering -- affinity matrix construction, spectral embedding, and $k$-means clustering -- using a single network in an end-to-end manner. BootSC leverages effective and efficient optimal-transport-derived supervision to bootstrap the affinity matrix and the cluster assignment matrix. Moreover, a semantically-consistent orthogonal re-parameterization technique is introduced to orthogonalize spectral embeddings, significantly enhancing the discrimination capability. Experimental results indicate that BootSC achieves state-of-the-art clustering performance. For example, it accomplishes a notable 16\% NMI improvement over the runner-up method on the challenging ImageNet-Dogs dataset. Our code is available at https://github.com/spdj2271/BootSC.