Sequential Disentanglement by Extracting Static Information From A Single Sequence Element

TL;DR

This paper introduces a novel architecture for unsupervised sequential disentanglement that effectively extracts static information from a single sequence element, reducing information leakage and improving generation and prediction performance across various data modalities.

Contribution

The paper proposes a simple, effective architecture with a subtraction inductive bias that mitigates information leakage in sequential disentanglement, simplifying the variational framework.

Findings

Achieves state-of-the-art results on multiple benchmarks.

Reduces complexity of loss terms and hyperparameters.

Effective across time series, video, and audio data.

Abstract

One of the fundamental representation learning tasks is unsupervised sequential disentanglement, where latent codes of inputs are decomposed to a single static factor and a sequence of dynamic factors. To extract this latent information, existing methods condition the static and dynamic codes on the entire input sequence. Unfortunately, these models often suffer from information leakage, i.e., the dynamic vectors encode both static and dynamic information, or vice versa, leading to a non-disentangled representation. Attempts to alleviate this problem via reducing the dynamic dimension and auxiliary loss terms gain only partial success. Instead, we propose a novel and simple architecture that mitigates information leakage by offering a simple and effective subtraction inductive bias while conditioning on a single sample. Remarkably, the resulting variational framework is simpler in terms of required loss terms, hyperparameters, and data augmentation. We evaluate our method on multiple data-modality benchmarks including general time series, video, and audio, and we show beyond state-of-the-art results on generation and prediction tasks in comparison to several strong baselines.