Orthogonal Polynomials Approximation Algorithm (OPAA):a functional analytic approach to estimating probability densities

TL;DR

OPAA is a novel parallelizable algorithm that estimates probability densities by transforming the problem into a functional space, enabling efficient computation of evidence and normalization weights.

Contribution

It introduces a functional analytic approach with a special transform for density estimation, providing a new scheme for computing evidence and normalization in Bayesian inference.

Findings

Parallelizable and efficient computation of density estimates

Accurate estimation of evidence and normalization weights

Applicable to Bayesian posterior analysis

Abstract

We present the new Orthogonal Polynomials Approximation Algorithm (OPAA), a parallelizable algorithm that estimates probability distributions using functional analytic approach: first, it finds a smooth functional estimate of the probability distribution, whether it is normalized or not; second, the algorithm provides an estimate of the normalizing weight; and third, the algorithm proposes a new computation scheme to compute such estimates. A core component of OPAA is a special transform of the square root of the joint distribution into a special functional space of our construct. Through this transform, the evidence is equated with the $L^2$ norm of the transformed function, squared. Hence, the evidence can be estimated by the sum of squares of the transform coefficients. Computations can be parallelized and completed in one pass. OPAA can be applied broadly to the estimation of probability density functions. In Bayesian problems, it can be applied to estimating the normalizing weight of the posterior, which is also known as the evidence, serving as an alternative to existing optimization-based methods.