Approximating Univariate Factored Distributions via Message-Passing Algorithms

TL;DR

This paper develops message-passing algorithms, including a novel VDBP method, to efficiently approximate univariate factored distributions like products of GMMs, addressing exponential complexity growth.

Contribution

It introduces a VDBP algorithm based on belief propagation for tractable approximation of factored PDFs and combines EP with new techniques to handle non-integrable beliefs.

Findings

VDBP effectively approximates univariate factored distributions.

The combined EP and new methods improve approximation robustness.

Algorithms handle non-integrable beliefs in complex models.

Abstract

Gaussian Mixture Models (GMMs) commonly arise in communication systems, particularly in bilinear joint estimation and detection problems. Although the product of GMMs is still a GMM, as the number of factors increases, the number of components in the resulting product GMM grows exponentially. To obtain a tractable approximation for a univariate factored probability density function (PDF), such as a product of GMMs, we investigate iterative message-passing algorithms. Based on Belief Propagation (BP), we propose a Variable Duplication and Gaussian Belief Propagation (VDBP)-based algorithm. The key idea of VDBP is to construct a multivariate measurement model whose marginal posterior is equal to the given univariate factored PDF. We then apply Gaussian BP (GaBP) to transform the global inference problem into local ones. Expectation propagation (EP) is another branch of message passing algorithms. In addition to converting the global approximation problem into local ones, it features a projection operation that ensures the intermediate functions (messages) belong to a desired family. Due to this projection, EP can be used to approximate the factored PDF directly. However, even if every factor is integrable, the division operation in EP may still cause the algorithm to fail when the mean and variance of a non-integrable belief are required. Therefore, this paper proposes two methods that combine EP with our previously proposed techniques for handling non-integrable beliefs to approximate univariate factored distributions.