New Advances and Theoretical Insights into EDML

TL;DR

This paper advances EDML for Bayesian networks by extending it to multivalued variables, simplifying its characterization, revealing its connection to EM, and proposing a hybrid algorithm with improved convergence.

Contribution

It introduces a multivalued extension of EDML, simplifies its characterization, links it to EM fixed points, and develops a hybrid EDML/EM algorithm.

Findings

Multivalued EDML extension developed

Simplified fixed-point characterization provided

Hybrid EDML/EM algorithm improves convergence

Abstract

EDML is a recently proposed algorithm for learning MAP parameters in Bayesian networks. In this paper, we present a number of new advances and insights on the EDML algorithm. First, we provide the multivalued extension of EDML, originally proposed for Bayesian networks over binary variables. Next, we identify a simplified characterization of EDML that further implies a simple fixed-point algorithm for the convex optimization problem that underlies it. This characterization further reveals a connection between EDML and EM: a fixed point of EDML is a fixed point of EM, and vice versa. We thus identify also a new characterization of EM fixed points, but in the semantics of EDML. Finally, we propose a hybrid EDML/EM algorithm that takes advantage of the improved empirical convergence behavior of EDML, while maintaining the monotonic improvement property of EM.