Density Ratio Estimation with Conditional Probability Paths

TL;DR

This paper introduces a new, efficient method for density ratio estimation in high-dimensional spaces using conditional probability paths, improving accuracy and speed over previous methods.

Contribution

The paper proposes a novel framework for time score estimation with a conditioning variable, enabling a closed-form objective and faster learning.

Findings

Faster learning of the time score compared to previous methods

Achieves competitive or superior density ratio estimation accuracy

Provides theoretical guarantees on estimation error

Abstract

Density ratio estimation in high dimensions can be reframed as integrating a certain quantity, the time score, over probability paths which interpolate between the two densities. In practice, the time score has to be estimated based on samples from the two densities. However, existing methods for this problem remain computationally expensive and can yield inaccurate estimates. Inspired by recent advances in generative modeling, we introduce a novel framework for time score estimation, based on a conditioning variable. Choosing the conditioning variable judiciously enables a closed-form objective function. We demonstrate that, compared to previous approaches, our approach results in faster learning of the time score and competitive or better estimation accuracies of the density ratio on challenging tasks. Furthermore, we establish theoretical guarantees on the error of the estimated density ratio.