Extending Contrastive Learning to Unsupervised Coreset Selection

TL;DR

This paper proposes an unsupervised coreset selection method based on contrastive learning, which reduces annotation costs and achieves comparable performance to supervised methods across multiple datasets.

Contribution

It introduces a novel unsupervised coreset selection approach leveraging contrastive learning similarity scores, enhancing efficiency without labeled data.

Findings

Unsupervised coreset selection outperforms random sampling.

Method achieves comparable results to supervised approaches on benchmark datasets.

Reduces the need for human annotation in data selection.

Abstract

Self-supervised contrastive learning offers a means of learning informative features from a pool of unlabeled data. In this paper, we delve into another useful approach -- providing a way of selecting a core-set that is entirely unlabeled. In this regard, contrastive learning, one of a large number of self-supervised methods, was recently proposed and has consistently delivered the highest performance. This prompted us to choose two leading methods for contrastive learning: the simple framework for contrastive learning of visual representations (SimCLR) and the momentum contrastive (MoCo) learning framework. We calculated the cosine similarities for each example of an epoch for the entire duration of the contrastive learning process and subsequently accumulated the cosine-similarity values to obtain the coreset score. Our assumption was that an sample with low similarity would likely behave as a coreset. Compared with existing coreset selection methods with labels, our approach reduced the cost associated with human annotation. The unsupervised method implemented in this study for coreset selection obtained improved results over a randomly chosen subset, and were comparable to existing supervised coreset selection on various classification datasets (e.g., CIFAR, SVHN, and QMNIST).