Context Tree Prior Distributions based on Node Weighting with exact Bayes Factors

TL;DR

This paper introduces a new way to specify prior distributions on context trees for variable-length Markov chains, enabling exact Bayesian model comparison and hypothesis testing with improved interpretability.

Contribution

It proposes a novel context-tree function-based prior that simplifies encoding structural beliefs and allows exact computation of marginal likelihoods and Bayes factors.

Findings

The new prior maintains computational tractability for exact inference.

Simulation studies show improved flexibility in modeling structural hypotheses.

Algorithms for model selection and depth determination are developed and validated.

Abstract

Variable-length Markov chains (VLMCs) are a flexible class of higher-order Markov models that admit a natural representation as context trees. Existing Bayesian methods for specifying prior distributions on tree structures rely on branching processes, but these suffer from a fundamental limitation. The connection between branching probabilities at individual nodes and the structural properties of the induced tree distribution is not straightforward, making it difficult to construct priors encoding specific structural beliefs. We address this limitation by introducing a novel representation of prior distributions on tree space based on context-tree functions. By directly specifying weights for individual contexts through a function on nodes, our approach provides an intuitive mechanism for incorporating structural hypotheses into the prior. This class of distributions maintains computational tractability, allowing marginal likelihoods and posterior mode trees to be computed exactly via generalizations of the Context Tree Weighting (CTW) and Context Tree Maximizing (CTM) algorithms. Exact Bayes factor computation enables rigorous model comparison and hypothesis testing. We demonstrate the flexibility and effectiveness of our approach through simulation studies comparing different prior specifications, and develop practical algorithms for selecting the maximal depth and performing model selection based on Bayes factors.