One-Step Score-Based Density Ratio Estimation

TL;DR

The paper introduces OS-DRE, a novel density ratio estimation method that combines accuracy and efficiency by decomposing the score into components and using an analytic RBF frame, enabling single evaluation inference.

Contribution

OS-DRE is a partly analytic, solver-free framework that improves density ratio estimation by removing the need for numerical solvers and enabling one-evaluation inference.

Findings

OS-DRE achieves a good balance between estimation quality and inference efficiency.

The method provides closed-form solutions for the temporal integral in density ratio estimation.

Experiments show OS-DRE performs well in density estimation, mutual information, and out-of-distribution detection.

Abstract

Density ratio estimation (DRE) is a useful tool for quantifying discrepancies between probability distributions, but existing approaches often involve a trade-off between estimation quality and computational efficiency. Classical direct DRE methods are usually efficient at inference time, yet their performance can seriously deteriorate when the discrepancy between distributions is large. In contrast, score-based DRE methods often yield more accurate estimates in such settings, but they typically require considerable repeated function evaluations and numerical integration. We propose One-step Score-based Density Ratio Estimation (OS-DRE), a partly analytic and solver-free framework designed to combine these complementary advantages. OS-DRE decomposes the time score into spatial and temporal components, representing the latter with an analytic radial basis function (RBF) frame. This formulation converts the otherwise intractable temporal integral into a closed-form weighted sum, thereby removing the need for numerical solvers and enabling DRE with only one function evaluation. We further analyze approximation conditions for the analytic frame, and establish approximation error bounds for both finitely and infinitely smooth temporal kernels, grounding the framework in existing approximation theory. Experiments across density estimation, continual Kullback-Leibler and mutual information estimation, and near out-of-distribution detection demonstrate that OS-DRE offers a favorable balance between estimation quality and inference efficiency.