Dequantified Diffusion-Schr{\"o}dinger Bridge for Density Ratio Estimation

TL;DR

This paper introduces D3RE, a novel framework combining diffusion and Schr{"o}dinger bridges with dequantization to improve density ratio estimation's robustness, stability, and efficiency, especially under challenging distributional differences.

Contribution

The paper proposes D3RE, a unified approach that leverages diffusion bridges and optimal transport to enhance density ratio estimation, addressing support coverage and score stability issues.

Findings

Outperforms baseline methods in mutual information estimation

Provides theoretical guarantees of uniform approximation and bounded scores

Demonstrates improved accuracy and stability in density estimation tasks

Abstract

Density ratio estimation is fundamental to tasks involving $f$-divergences, yet existing methods often fail under significantly different distributions or inadequately overlapping supports -- the density-chasm and the support-chasm problems. Additionally, prior approaches yield divergent time scores near boundaries, leading to instability. We design $\textbf{D}^3\textbf{RE}$, a unified framework for \textbf{robust}, \textbf{stable} and \textbf{efficient} density ratio estimation. We propose the dequantified diffusion bridge interpolant (DDBI), which expands support coverage and stabilizes time scores via diffusion bridges and Gaussian dequantization. Building on DDBI, the proposed dequantified Schr{\"o}dinger bridge interpolant (DSBI) incorporates optimal transport to solve the Schr{\"o}dinger bridge problem, enhancing accuracy and efficiency. Our method offers uniform approximation and bounded time scores in theory, and outperforms baselines empirically in mutual information and density estimation tasks.