Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifolds

TL;DR

This paper introduces a geometric approach to density ratio estimation using generalized geodesics on statistical manifolds, improving stability and accuracy over traditional incremental mixture methods.

Contribution

It reinterprets existing methods geometrically and proposes a new iterative algorithm for sampling along geodesics, enhancing density ratio estimation.

Findings

Outperforms existing incremental mixture methods in accuracy

Sampling along geodesics reduces estimation variance

Changing geodesic distances affects estimation stability

Abstract

The density ratio of two probability distributions is one of the fundamental tools in mathematical and computational statistics and machine learning, and it has a variety of known applications. Therefore, density ratio estimation from finite samples is a very important task, but it is known to be unstable when the distributions are distant from each other. One approach to address this problem is density ratio estimation using incremental mixtures of the two distributions. We geometrically reinterpret existing methods for density ratio estimation based on incremental mixtures. We show that these methods can be regarded as iterating on the Riemannian manifold along a particular curve between the two probability distributions. Making use of the geometry of the manifold, we propose to consider incremental density ratio estimation along generalized geodesics on this manifold. To achieve such a method requires Monte Carlo sampling along geodesics via transformations of the two distributions. We show how to implement an iterative algorithm to sample along these geodesics and show how changing the distances along the geodesic affect the variance and accuracy of the estimation of the density ratio. Our experiments demonstrate that the proposed approach outperforms the existing approaches using incremental mixtures that do not take the geometry of the