Life-Long Disentangled Representation Learning with Cross-Domain Latent Homologies

TL;DR

VASE is a novel unsupervised learning algorithm that detects data shifts, prevents forgetting, and learns disentangled, semantically meaningful representations across domains, enabling cross-domain inference.

Contribution

The paper introduces VASE, a new method for lifelong unsupervised representation learning that automatically detects distribution shifts and encourages disentangled, cross-domain latent sharing.

Findings

VASE effectively detects data distribution shifts.

It prevents catastrophic forgetting of previous knowledge.

It enables semantically meaningful cross-domain inference.

Abstract

Intelligent behaviour in the real-world requires the ability to acquire new knowledge from an ongoing sequence of experiences while preserving and reusing past knowledge. We propose a novel algorithm for unsupervised representation learning from piece-wise stationary visual data: Variational Autoencoder with Shared Embeddings (VASE). Based on the Minimum Description Length principle, VASE automatically detects shifts in the data distribution and allocates spare representational capacity to new knowledge, while simultaneously protecting previously learnt representations from catastrophic forgetting. Our approach encourages the learnt representations to be disentangled, which imparts a number of desirable properties: VASE can deal sensibly with ambiguous inputs, it can enhance its own representations through imagination-based exploration, and most importantly, it exhibits semantically meaningful sharing of latents between different datasets. Compared to baselines with entangled representations, our approach is able to reason beyond surface-level statistics and perform semantically meaningful cross-domain inference.