Disentanglement in Difference: Directly Learning Semantically Disentangled Representations by Maximizing Inter-Factor Differences

TL;DR

This paper introduces Disentanglement in Difference (DiD), a novel approach that directly learns semantic differences in representations, outperforming traditional methods that focus on statistical independence.

Contribution

DiD directly models semantic differences using a Difference Encoder and contrastive loss, addressing the gap between statistical independence and semantic disentanglement.

Findings

DiD outperforms existing methods on dSprites and 3DShapes datasets.

DiD achieves higher scores on various disentanglement metrics.

The approach effectively differentiates semantic factors in learned representations.

Abstract

In this study, Disentanglement in Difference(DiD) is proposed to address the inherent inconsistency between the statistical independence of latent variables and the goal of semantic disentanglement in disentanglement representation learning. Conventional disentanglement methods achieve disentanglement representation by improving statistical independence among latent variables. However, the statistical independence of latent variables does not necessarily imply that they are semantically unrelated, thus, improving statistical independence does not always enhance disentanglement performance. To address the above issue, DiD is proposed to directly learn semantic differences rather than the statistical independence of latent variables. In the DiD, a Difference Encoder is designed to measure the semantic differences; a contrastive loss function is established to facilitate inter-dimensional comparison. Both of them allow the model to directly differentiate and disentangle distinct semantic factors, thereby resolving the inconsistency between statistical independence and semantic disentanglement. Experimental results on the dSprites and 3DShapes datasets demonstrate that the proposed DiD outperforms existing mainstream methods across various disentanglement metrics.