Mixtures of Gaussians and Minimum Relative Entropy Techniques for Modeling Continuous Uncertainties

TL;DR

This paper presents a method for modeling continuous uncertainties using mixtures of Gaussians and minimum relative entropy techniques, enhancing probabilistic inference accuracy and computational efficiency.

Contribution

It introduces a novel approach combining transformations and Gaussian mixtures driven by minimum relative entropy, with automated component selection using EM algorithm.

Findings

Effective transformation and fitting of continuous variables

Automated selection of mixture components balancing accuracy and cost

Use of influence diagram methods for efficient inference

Abstract

Problems of probabilistic inference and decision making under uncertainty commonly involve continuous random variables. Often these are discretized to a few points, to simplify assessments and computations. An alternative approximation is to fit analytically tractable continuous probability distributions. This approach has potential simplicity and accuracy advantages, especially if variables can be transformed first. This paper shows how a minimum relative entropy criterion can drive both transformation and fitting, illustrating with a power and logarithm family of transformations and mixtures of Gaussian (normal) distributions, which allow use of efficient influence diagram methods. The fitting procedure in this case is the well-known EM algorithm. The selection of the number of components in a fitted mixture distribution is automated with an objective that trades off accuracy and computational cost.