Latent Dependency Forest Models

TL;DR

Latent dependency forest models (LDFMs) are a new probabilistic framework that dynamically models variable dependencies with a forest structure, capturing context-specific independence and avoiding complex structure learning.

Contribution

We introduce LDFMs, a novel probabilistic model parameterized by dependency grammars, simplifying learning and enabling dynamic dependency modeling.

Findings

LDFMs are competitive with existing probabilistic models.

They effectively model context-specific independence.

Learning reduces to parameter estimation, bypassing structure learning.

Abstract

Probabilistic modeling is one of the foundations of modern machine learning and artificial intelligence. In this paper, we propose a novel type of probabilistic models named latent dependency forest models (LDFMs). A LDFM models the dependencies between random variables with a forest structure that can change dynamically based on the variable values. It is therefore capable of modeling context-specific independence. We parameterize a LDFM using a first-order non-projective dependency grammar. Learning LDFMs from data can be formulated purely as a parameter learning problem, and hence the difficult problem of model structure learning is circumvented. Our experimental results show that LDFMs are competitive with existing probabilistic models.