Simple Disentanglement of Style and Content in Visual Representations

TL;DR

This paper introduces a simple post-processing method to disentangle style and content in pre-trained visual representations, improving domain generalization under style-related distribution shifts.

Contribution

It proposes a probabilistic linear model and a straightforward algorithm for disentangling style and content, applicable to large-scale datasets like ImageNet.

Findings

Provably disentangles style and content features

Enhances domain generalization performance

Effective on large-scale vision datasets

Abstract

Learning visual representations with interpretable features, i.e., disentangled representations, remains a challenging problem. Existing methods demonstrate some success but are hard to apply to large-scale vision datasets like ImageNet. In this work, we propose a simple post-processing framework to disentangle content and style in learned representations from pre-trained vision models. We model the pre-trained features probabilistically as linearly entangled combinations of the latent content and style factors and develop a simple disentanglement algorithm based on the probabilistic model. We show that the method provably disentangles content and style features and verify its efficacy empirically. Our post-processed features yield significant domain generalization performance improvements when the distribution shift occurs due to style changes or style-related spurious correlations.