Probabilistic Inference from Arbitrary Uncertainty using Mixtures of Factorized Generalized Gaussians

TL;DR

This paper introduces a versatile probabilistic inference framework using mixtures of factorized generalized Gaussians, enabling direct computation of conditional distributions from uncertain data without numerical integration.

Contribution

It develops a novel mixture model approach with an extended EM algorithm for efficient inference and learning from arbitrary uncertain information, applicable across various fields.

Findings

Effective in nonparametric pattern classification

Improves nonlinear regression accuracy

Enhances pattern completion robustness

Abstract

This paper presents a general and efficient framework for probabilistic inference and learning from arbitrary uncertain information. It exploits the calculation properties of finite mixture models, conjugate families and factorization. Both the joint probability density of the variables and the likelihood function of the (objective or subjective) observation are approximated by a special mixture model, in such a way that any desired conditional distribution can be directly obtained without numerical integration. We have developed an extended version of the expectation maximization (EM) algorithm to estimate the parameters of mixture models from uncertain training examples (indirect observations). As a consequence, any piece of exact or uncertain information about both input and output values is consistently handled in the inference and learning stages. This ability, extremely useful in certain situations, is not found in most alternative methods. The proposed framework is formally justified from standard probabilistic principles and illustrative examples are provided in the fields of nonparametric pattern classification, nonlinear regression and pattern completion. Finally, experiments on a real application and comparative results over standard databases provide empirical evidence of the utility of the method in a wide range of applications.