A Comparison of Algorithms for Learning Hidden Variables in Normal Graphs

TL;DR

This paper compares different algorithms for learning hidden variables in normal-form Bayesian factor graphs, providing explicit methods and performance evaluations on synthetic data.

Contribution

It introduces explicit localized adaptation algorithms derived from ML and KL-divergence criteria for Bayesian graphs in normal form.

Findings

Algorithms verified on synthetic data sets

Comparison of ML-based, Viterbi-like, and variational algorithms

Performance insights for different architectures

Abstract

A Bayesian factor graph reduced to normal form consists in the interconnection of diverter units (or equal constraint units) and Single-Input/Single-Output (SISO) blocks. In this framework localized adaptation rules are explicitly derived from a constrained maximum likelihood (ML) formulation and from a minimum KL-divergence criterion using KKT conditions. The learning algorithms are compared with two other updating equations based on a Viterbi-like and on a variational approximation respectively. The performance of the various algorithm is verified on synthetic data sets for various architectures. The objective of this paper is to provide the programmer with explicit algorithms for rapid deployment of Bayesian graphs in the applications.