Mean Field Variational Approximation for Continuous-Time Bayesian Networks

TL;DR

This paper introduces a mean field variational method for continuous-time Bayesian networks that enables efficient inference and learning by approximating complex distributions with a product of simpler Markov processes.

Contribution

It develops a novel variational approximation framework for CTBNs, providing theoretical foundations, efficient ODE-based implementation, and practical applications.

Findings

Provides a globally consistent distribution approximation

Offers a lower bound on observation probabilities for learning

Demonstrates effectiveness on large-scale real-world problems

Abstract

Continuous-time Bayesian networks is a natural structured representation language for multicomponent stochastic processes that evolve continuously over time. Despite the compact representation, inference in such models is intractable even in relatively simple structured networks. Here we introduce a mean field variational approximation in which we use a product of inhomogeneous Markov processes to approximate a distribution over trajectories. This variational approach leads to a globally consistent distribution, which can be efficiently queried. Additionally, it provides a lower bound on the probability of observations, thus making it attractive for learning tasks. We provide the theoretical foundations for the approximation, an efficient implementation that exploits the wide range of highly optimized ordinary differential equations (ODE) solvers, experimentally explore characterizations of processes for which this approximation is suitable, and show applications to a large-scale realworld inference problem.