Joint estimation of intersecting context tree models

TL;DR

This paper introduces a joint estimation method for intersecting context tree models from two sources, using a BIC-penalized likelihood approach, with proven consistency and demonstrated practical benefits.

Contribution

It proposes a novel joint estimation algorithm for intersecting context trees, improving over separate methods, with theoretical consistency guarantees.

Findings

The joint estimator is strongly consistent.

The proposed algorithm is computationally efficient.

Simulation results show advantages over separate estimation methods.

Abstract

We study a problem of model selection for data produced by two different context tree sources. Motivated by linguistic questions, we consider the case where the probabilistic context trees corresponding to the two sources are finite and share many of their contexts. In order to understand the differences between the two sources, it is important to identify which contexts and which transition probabilities are specific to each source. We consider a class of probabilistic context tree models with three types of contexts: those which appear in one, the other, or both sources. We use a BIC penalized maximum likelihood procedure that jointly estimates the two sources. We propose a new algorithm which efficiently computes the estimated context trees. We prove that the procedure is strongly consistent. We also present a simulation study showing the practical advantage of our procedure over a procedure that works separately on each dataset.