Pathwise guessing in categorical time series with unbounded alphabets

TL;DR

This paper introduces a non-parametric guessing method for categorical time series that achieves near-optimal learning rates independent of alphabet size, applicable to various models like Markov chains and Gibbs measures.

Contribution

It proposes a novel guessing function with a universal learning rate and provides theoretical bounds demonstrating its near-optimality across broad time series models.

Findings

Achieves a learning rate independent of alphabet size.

Provides a minimax lower bound matching the estimator's upper bound.

Applicable to diverse models including Markov, hidden Markov, and Gibbs measures.

Abstract

The following learning problem arises naturally in various applications: Given a finite sample from a categorical or count time series, can we learn a function of the sample that (nearly) maximizes the probability of correctly guessing the values of a given portion of the data using the values from the remaining parts? Unlike classical approaches in statistical inference, our approach avoids explicitly estimating the conditional probabilities. We propose a non-parametric guessing function with a learning rate independent of the alphabet size. Our analysis focuses on a broad class of time series models that encompasses finite-order Markov chains, some hidden Markov chains, Poisson regression for count processes, and one-dimensional Gibbs measures. We provide a margin condition that controls the rate of convergence for the risk. Additionally, we establish a minimax lower bound for the convergence rate of the risk associated with our guessing problem. This lower bound matches the upper bound achieved by our estimator up to a logarithmic factor, demonstrating its near-optimality.