Structured Contrastive Learning for Interpretable Latent Representations

TL;DR

This paper introduces Structured Contrastive Learning (SCL), a novel framework that partitions latent space into interpretable groups, significantly improving robustness and interpretability of neural networks against transformations like phase shifts and rotations.

Contribution

The paper proposes SCL, a framework that explicitly structures latent spaces into semantic groups, enhancing robustness and interpretability without architectural changes.

Findings

ECG similarity improved from 0.25 to 0.91 under phase shifts

IMU rotation robustness achieved 86.65% accuracy

Outperforms traditional data augmentation methods

Abstract

Neural networks exhibit severe brittleness to semantically irrelevant transformations. A mere 75ms electrocardiogram (ECG) phase shift degrades latent cosine similarity from 1.0 to 0.2, while sensor rotations collapse activity recognition performance with inertial measurement units (IMUs). We identify the root cause as "laissez-faire" representation learning, where latent spaces evolve unconstrained provided task performance is satisfied. We propose Structured Contrastive Learning (SCL), a framework that partitions latent space representations into three semantic groups: invariant features that remain consistent under given transformations (e.g., phase shifts or rotations), variant features that actively differentiate transformations via a novel variant mechanism, and free features that preserve task flexibility. This creates controllable push-pull dynamics where different latent dimensions serve distinct, interpretable purposes. The variant mechanism enhances contrastive learning by encouraging variant features to differentiate within positive pairs, enabling simultaneous robustness and interpretability. Our approach requires no architectural modifications and integrates seamlessly into existing training pipelines. Experiments on ECG phase invariance and IMU rotation robustness demonstrate superior performance: ECG similarity improves from 0.25 to 0.91 under phase shifts, while WISDM activity recognition achieves 86.65% accuracy with 95.38% rotation consistency, consistently outperforming traditional data augmentation. This work represents a paradigm shift from reactive data augmentation to proactive structural learning, enabling interpretable latent representations in neural networks.