Selective Pseudo-label Clustering

TL;DR

This paper introduces a selective pseudo-label clustering method that trains DNNs using only the most confident pseudo-labels, improving clustering performance on image datasets.

Contribution

It proposes a novel selective pseudo-label approach that filters pseudo-labels to enhance DNN training for clustering tasks, with formal performance guarantees.

Findings

Achieves state-of-the-art results on three image datasets

Proves performance gains under certain conditions

Reduces noise impact in pseudo-label training

Abstract

Deep neural networks (DNNs) offer a means of addressing the challenging task of clustering high-dimensional data. DNNs can extract useful features, and so produce a lower dimensional representation, which is more amenable to clustering techniques. As clustering is typically performed in a purely unsupervised setting, where no training labels are available, the question then arises as to how the DNN feature extractor can be trained. The most accurate existing approaches combine the training of the DNN with the clustering objective, so that information from the clustering process can be used to update the DNN to produce better features for clustering. One problem with this approach is that these ``pseudo-labels'' produced by the clustering algorithm are noisy, and any errors that they contain will hurt the training of the DNN. In this paper, we propose selective pseudo-label clustering, which uses only the most confident pseudo-labels for training the~DNN. We formally prove the performance gains under certain conditions. Applied to the task of image clustering, the new approach achieves a state-of-the-art performance on three popular image datasets. Code is available at https://github.com/Lou1sM/clustering.