A Distribution Similarity Based Regularizer for Learning Bayesian Networks

TL;DR

This paper introduces a novel distribution similarity regularizer for Bayesian networks that improves modeling of wave propagation in inhomogeneous media by encouraging similar conditional distributions, leading to better generalization.

Contribution

It proposes a new distribution-based penalization method for Bayesian networks that leverages high-level similarities among factors, enhancing model regularization.

Findings

The proposed method effectively models wave propagation in complex media.

It outperforms baseline methods in the specific application.

The approach improves generalization in probabilistic graphical models.

Abstract

Probabilistic graphical models compactly represent joint distributions by decomposing them into factors over subsets of random variables. In Bayesian networks, the factors are conditional probability distributions. For many problems, common information exists among those factors. Adding similarity restrictions can be viewed as imposing prior knowledge for model regularization. With proper restrictions, learned models usually generalize better. In this work, we study methods that exploit such high-level similarities to regularize the learning process and apply them to the task of modeling the wave propagation in inhomogeneous media. We propose a novel distribution-based penalization approach that encourages similar conditional probability distribution rather than force the parameters to be similar explicitly. We show in experiment that our proposed algorithm solves the modeling wave propagation problem, which other baseline methods are not able to solve.