Marginal Inference queries in Hidden Markov Models under context-free grammar constraints

TL;DR

This paper develops algorithms for computing and approximating the likelihood of context-free grammars within Hidden Markov Models, addressing computational complexity and providing practical approximation methods.

Contribution

It introduces a dynamic algorithm for exact likelihood computation in unambiguous CFGs and proposes an FPRAS for ambiguous CFGs, advancing inference in sequential probabilistic models.

Findings

Exact likelihood computation is NP-Hard for ambiguous CFGs.

A polynomial-time randomized approximation scheme is proposed for certain CFGs.

The algorithms enable better inference in NLP and computational linguistics applications.

Abstract

The primary use of any probabilistic model involving a set of random variables is to run inference and sampling queries on it. Inference queries in classical probabilistic models is concerned by the computation of marginal or conditional probabilities of events given as an input. When the probabilistic model is sequential, more sophisticated marginal inference queries involving complex grammars may be of interest in fields such as computational linguistics and NLP. In this work, we address the question of computing the likelihood of context-free grammars (CFGs) in Hidden Markov Models (HMMs). We provide a dynamic algorithm for the exact computation of the likelihood for the class of unambiguous context-free grammars. We show that the problem is NP-Hard, even with the promise that the input CFG has a degree of ambiguity less than or equal to 2. We then propose a fully polynomial randomized approximation scheme (FPRAS) algorithm to approximate the likelihood for the case of polynomially-bounded ambiguous CFGs.