Information based Deep Clustering: An experimental study

TL;DR

This paper provides a comprehensive experimental analysis of deep clustering methods based on mutual information, comparing different transformations and loss functions to identify optimal combinations for improved clustering performance.

Contribution

It systematically evaluates the interactions of various transformations and loss functions in mutual information-based deep clustering, highlighting the effectiveness of combined geometrical and adversarial transformations.

Findings

Mutual information between a sample and its transformed version yields state-of-the-art results.

Combining geometrical and adversarial transformations enhances clustering performance.

Different loss functions and transformations interact in ways that significantly impact results.

Abstract

Recently, two methods have shown outstanding performance for clustering images and jointly learning the feature representation. The first, called Information Maximiz-ing Self-Augmented Training (IMSAT), maximizes the mutual information between input and clusters while using a regularization term based on virtual adversarial examples. The second, named Invariant Information Clustering (IIC), maximizes the mutual information between the clustering of a sample and its geometrically transformed version. These methods use mutual information in distinct ways and leverage different kinds of transformations. This work proposes a comprehensive analysis of transformation and losses for deep clustering, where we compare numerous combinations of these two components and evaluate how they interact with one another. Results suggest that mutual information between a sample and its transformed representation leads to state-of-the-art performance for deep clustering, especially when used jointly with geometrical and adversarial transformations.