Structure Learning and Parameter Estimation for Graphical Models via Penalized Maximum Likelihood Methods

TL;DR

This paper explores structure learning and parameter estimation in probabilistic graphical models, specifically Bayesian networks and continuous time Bayesian networks, using penalized maximum likelihood methods with LASSO, addressing both complete and incomplete data scenarios.

Contribution

It introduces a unified LASSO-based maximum likelihood framework for structure learning in BNs and CTBNs, including theoretical analysis and experimental validation.

Findings

LASSO penalty effectively identifies true model structure.

Method performs well with both complete and incomplete data.

Theoretical results support practical applicability.

Abstract

Probabilistic graphical models (PGMs) provide a compact and flexible framework to model very complex real-life phenomena. They combine the probability theory which deals with uncertainty and logical structure represented by a graph which allows one to cope with the computational complexity and also interpret and communicate the obtained knowledge. In the thesis, we consider two different types of PGMs: Bayesian networks (BNs) which are static, and continuous time Bayesian networks which, as the name suggests, have a temporal component. We are interested in recovering their true structure, which is the first step in learning any PGM. This is a challenging task, which is interesting in itself from the causal point of view, for the purposes of interpretation of the model and the decision-making process. All approaches for structure learning in the thesis are united by the same idea of maximum likelihood estimation with the LASSO penalty. The problem of structure learning is reduced to the problem of finding non-zero coefficients in the LASSO estimator for a generalized linear model. In the case of CTBNs, we consider the problem both for complete and incomplete data. We support the theoretical results with experiments.