Direct Coloring for Self-Supervised Enhanced Feature Decoupling

TL;DR

This paper introduces a novel feature decoupling method called direct coloring, which enhances self-supervised learning by promoting useful features early on, outperforming existing contrastive and non-contrastive approaches.

Contribution

It proposes a new feature decoupling technique based on Bayesian-informed coloring, providing a complementary and more effective alternative to whitening in SSL.

Findings

Outperforms contrastive and non-contrastive methods

Enhances feature decoupling without collapse

Complementary to existing SSL techniques

Abstract

The success of self-supervised learning (SSL) has been the focus of multiple recent theoretical and empirical studies, including the role of data augmentation (in feature decoupling) as well as complete and dimensional representation collapse. While complete collapse is well-studied and addressed, dimensional collapse has only gain attention and addressed in recent years mostly using variants of redundancy reduction (aka whitening) techniques. In this paper, we further explore a complementary approach to whitening via feature decoupling for improved representation learning while avoiding representation collapse. In particular, we perform feature decoupling by early promotion of useful features via careful feature coloring. The coloring technique is developed based on a Bayesian prior of the augmented data, which is inherently encoded for feature decoupling. We show that our proposed framework is complementary to the state-of-the-art techniques, while outperforming both contrastive and recent non-contrastive methods. We also study the different effects of coloring approach to formulate it as a general complementary technique along with other baselines.