Efficient Learning of Optimal Markov Network Topology with k-Tree Modeling

TL;DR

This paper extends the classic Chow-Liu algorithm to learn optimal Markov network topologies with bounded tree width, providing polynomial algorithms under certain conditions for systems with higher-order correlations.

Contribution

It generalizes the maximum spanning tree approach to maximum spanning k-trees for Markov networks of higher tree width, with efficient algorithms under specific conditions.

Findings

Proves the optimality of maximum spanning k-trees for Markov network approximation.

Develops polynomial algorithms for learning higher-order correlation structures.

Provides conditions under which the algorithms are computationally feasible.

Abstract

The seminal work of Chow and Liu (1968) shows that approximation of a finite probabilistic system by Markov trees can achieve the minimum information loss with the topology of a maximum spanning tree. Our current paper generalizes the result to Markov networks of tree width $\leq k$, for every fixed $k\geq 2$. In particular, we prove that approximation of a finite probabilistic system with such Markov networks has the minimum information loss when the network topology is achieved with a maximum spanning $k$-tree. While constructing a maximum spanning $k$-tree is intractable for even $k=2$, we show that polynomial algorithms can be ensured by a sufficient condition accommodated by many meaningful applications. In particular, we prove an efficient algorithm for learning the optimal topology of higher order correlations among random variables that belong to an underlying linear structure.