Conditional Distribution Quantization in Machine Learning

TL;DR

This paper introduces a novel method using n-point conditional quantizations to better model complex multimodal conditional distributions in machine learning, surpassing traditional expectation-based approaches.

Contribution

It develops a learnable, gradient-based quantization framework tailored for conditional distributions, enabling multimodal data modeling and uncertainty quantification.

Findings

Effective in capturing multimodal structures

Improves uncertainty quantification in predictions

Demonstrated on synthetic and real datasets

Abstract

Conditional expectation \mathbb{E}(Y \mid X) often fails to capture the complexity of multimodal conditional distributions \mathcal{L}(Y \mid X). To address this, we propose using n-point conditional quantizations--functional mappings of X that are learnable via gradient descent--to approximate \mathcal{L}(Y \mid X). This approach adapts Competitive Learning Vector Quantization (CLVQ), tailored for conditional distributions. It goes beyond single-valued predictions by providing multiple representative points that better reflect multimodal structures. It enables the approximation of the true conditional law in the Wasserstein distance. The resulting framework is theoretically grounded and useful for uncertainty quantification and multimodal data generation tasks. For example, in computer vision inpainting tasks, multiple plausible reconstructions may exist for the same partially observed input image X. We demonstrate the effectiveness of our approach through experiments on synthetic and real-world datasets.