Self-Supervised Representation Learning via Hyperspherical Density Shaping

TL;DR

HyDeS is a theoretically grounded self-supervised learning method that maximizes mutual information in hyperspherical space, improving segmentation but less so for fine-grained classification.

Contribution

It introduces HyDeS, a novel approach based on hyperspherical density shaping and non-parametric von Mises-Fisher density estimation for self-supervised learning.

Findings

HyDeS biases models towards foreground features.

Performs well on segmentation tasks like VOC PASCAL.

Lagging in fine-grained classification.

Abstract

Modern self-supervised representation learning methods often relies on empirical heuristics that are not theoretically grounded. In this study we propose HyDeS, a theoretically grounded method based on multi-view mutual information maximization within an hyperspherical space using Shannon differential entropy with a non-parametric von Mises-Fisher density estimator. We show that HyDeS bias the trained model towards focusing on foreground features of the images and perform well on segmentation tasks such as VOC PASCAL, while it lags in fine-grained classification. We provide a detailed analysis of the induced latent space geometry and learning dynamics, that can be used for designing other theoretically grounded self-supervised learning methods.