Self-Supervised Learning as Discrete Communication

TL;DR

This paper introduces a novel self-supervised learning framework that uses discrete binary communication between networks to improve structured representation learning and semantic encoding, outperforming traditional continuous methods.

Contribution

It proposes framing SSL as a discrete communication process with binary messages, enhancing representation structure and semantic encoding over continuous approaches.

Findings

Improved performance on image classification, retrieval, and dense prediction tasks.

Binary codes form a compact, semantic language capturing reusable factors.

Enhanced robustness under domain shift through self-supervised adaptation.

Abstract

Most self-supervised learning (SSL) methods learn continuous visual representations by aligning different views of the same input, offering limited control over how information is structured across representation dimensions. In this work, we frame visual self-supervised learning as a discrete communication process between a teacher and a student network, where semantic information is transmitted through a fixed-capacity binary channel. Rather than aligning continuous features, the student predicts multi-label binary messages produced by the teacher. Discrete agreement is enforced through an element-wise binary cross-entropy objective, while a coding-rate regularization term encourages effective utilization of the constrained channel, promoting structured representations. We further show that periodically reinitializing the projection head strengthens this effect by encouraging embeddings that remain predictive across multiple discrete encodings. Extensive experiments demonstrate consistent improvements over continuous agreement baselines on image classification, retrieval, and dense visual prediction tasks, as well as under domain shift through self-supervised adaptation. Beyond backbone representations, we analyze the learned binary codes and show that they form a compact and informative discrete language, capturing semantic factors reusable across classes.