SuperCM: Improving Semi-Supervised Learning and Domain Adaptation through differentiable clustering

TL;DR

SuperCM introduces a differentiable clustering module for semi-supervised learning and domain adaptation, explicitly leveraging cluster structures to improve performance, especially with limited labeled data, through an end-to-end training approach.

Contribution

It proposes a novel differentiable clustering method that explicitly incorporates cluster information into SSL and UDA, enhancing effectiveness over existing implicit methods.

Findings

Effective in low supervision regimes

Improves performance as a standalone model

Serves as a regularizer for existing approaches

Abstract

Semi-Supervised Learning (SSL) and Unsupervised Domain Adaptation (UDA) enhance the model performance by exploiting information from labeled and unlabeled data. The clustering assumption has proven advantageous for learning with limited supervision and states that data points belonging to the same cluster in a high-dimensional space should be assigned to the same category. Recent works have utilized different training mechanisms to implicitly enforce this assumption for the SSL and UDA. In this work, we take a different approach by explicitly involving a differentiable clustering module which is extended to leverage the supervised data to compute its centroids. We demonstrate the effectiveness of our straightforward end-to-end training strategy for SSL and UDA over extensive experiments and highlight its benefits, especially in low supervision regimes, both as a standalone model and as a regularizer for existing approaches.