Zero-Flow Encoders

TL;DR

This paper introduces a flow-inspired framework that uses a zero-flow criterion to certify conditional independence and learn meaningful representations in generative models, self-supervised learning, and graphical models.

Contribution

It proposes a novel zero-flow criterion based on flow models to certify conditional independence and extract sufficient data information, enabling new representation learning methods.

Findings

Zero-flow criterion certifies conditional independence.

The framework effectively learns representations in various tasks.

Experimental results validate the approach on real-world datasets.

Abstract

Flow-based methods have achieved significant success in various generative modeling tasks, capturing nuanced details within complex data distributions. However, few existing works have exploited this unique capability to resolve fine-grained structural details beyond generation tasks. This paper presents a flow-inspired framework for representation learning. First, we demonstrate that a rectified flow trained using independent coupling is zero everywhere at $t=0.5$ if and only if the source and target distributions are identical. We term this property the \emph{zero-flow criterion}. Second, we show that this criterion can certify conditional independence, thereby extracting \emph{sufficient information} from the data. Third, we translate this criterion into a tractable, simulation-free loss function that enables learning amortized Markov blankets in graphical models and latent representations in self-supervised learning tasks. Experiments on both simulated and real-world datasets demonstrate the effectiveness of our approach. The code reproducing our experiments can be found at: https://github.com/probabilityFLOW/zfe.