Diffusion Density Estimators

TL;DR

This paper presents a novel, highly parallelizable method for neural density estimation using diffusion models, eliminating the need for flow-based ODE solving and improving scalability and efficiency.

Contribution

It introduces a Monte Carlo-based approach to estimate log densities directly, bypassing the traditional Probability Flow ODE method, and analyzes training parameters for better accuracy.

Findings

The new method is highly parallelizable and faster.

Training parameters significantly impact density estimation accuracy.

The approach enhances scalability and efficiency of diffusion-based density estimators.

Abstract

We investigate the use of diffusion models as neural density estimators. The current approach to this problem involves converting the generative process to a smooth flow, known as the Probability Flow ODE. The log density at a given sample can be obtained by solving the ODE with a black-box solver. We introduce a new, highly parallelizable method that computes log densities without the need to solve a flow. Our approach is based on estimating a path integral by Monte Carlo, in a manner identical to the simulation-free training of diffusion models. We also study how different training parameters affect the accuracy of the density calculation, and offer insights into how these models can be made more scalable and efficient.