Semi-Discrete Normalizing Flows through Differentiable Tessellation

TL;DR

This paper introduces a novel tessellation-based normalizing flow method that learns quantization boundaries in continuous space, enabling efficient and exact likelihood evaluation for mapping between discrete and continuous distributions.

Contribution

It proposes a differentiable tessellation approach with normalizing flows on convex polytopes, allowing direct learning of quantization boundaries and efficient likelihood computation.

Findings

Improves likelihood evaluation efficiency in discrete-continuous mappings.

Demonstrates superior performance over existing methods on structured data.

Enables encoding of structural relations through learned quantization boundaries.

Abstract

Mapping between discrete and continuous distributions is a difficult task and many have had to resort to heuristical approaches. We propose a tessellation-based approach that directly learns quantization boundaries in a continuous space, complete with exact likelihood evaluations. This is done through constructing normalizing flows on convex polytopes parameterized using a simple homeomorphism with an efficient log determinant Jacobian. We explore this approach in two application settings, mapping from discrete to continuous and vice versa. Firstly, a Voronoi dequantization allows automatically learning quantization boundaries in a multidimensional space. The location of boundaries and distances between regions can encode useful structural relations between the quantized discrete values. Secondly, a Voronoi mixture model has near-constant computation cost for likelihood evaluation regardless of the number of mixture components. Empirically, we show improvements over existing methods across a range of structured data modalities.