Entropy Message Passing
Velimir M. Ilic, Miomir S. Stankovic, Branimir T. Todorovic

TL;DR
The paper introduces entropy message passing (EMP), a novel algorithm for cycle-free factor graphs that efficiently computes entropy and related expressions, enhancing inference methods in probabilistic models.
Contribution
It presents EMP as a new message passing algorithm based on the entropy semiring, extending sum-product algorithms to compute entropy and expectations in probabilistic models.
Findings
EMP efficiently computes entropy in cycle-free factor graphs.
EMP can be applied to expectation maximization and gradient descent.
The algorithm bridges automata theory and probabilistic inference.
Abstract
The paper proposes a new message passing algorithm for cycle-free factor graphs. The proposed "entropy message passing" (EMP) algorithm may be viewed as sum-product message passing over the entropy semiring, which has previously appeared in automata theory. The primary use of EMP is to compute the entropy of a model. However, EMP can also be used to compute expressions that appear in expectation maximization and in gradient descent algorithms.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
