Maximum entropy estimation of probability distributions with Gaussian conditions

TL;DR

This paper introduces a maximum entropy-based method for estimating univariate probability distributions using Gaussian local conditions, improving local accuracy and suitable for heuristic optimization, with demonstrated effectiveness on large datasets.

Contribution

It presents a novel approach combining maximum entropy with Gaussian local conditions for probability density estimation, enhancing local accuracy over classical methods.

Findings

Estimation improves with larger sample sizes, typically over 1,000 values.

Local conditions enhance local estimation accuracy.

The method is effective and compatible with heuristic optimization techniques.

Abstract

We describe a method to computationally estimate the probability density function of a univariate random variable by applying the maximum entropy principle with some local conditions given by Gaussian functions. The estimation errors and optimal values of parameters are determined. Experimental results are presented. The method estimates the distribution well if a large enough selection is used, typically at least 1 000 values. Compared to the classical approach of entropy maximisation, local conditions allow improving estimation locally. The method is well suited for a heuristic optimisation approach.