Exploring Predictive States via Cantor Embeddings and Wasserstein Distance

TL;DR

This paper introduces a novel approach to analyze predictive states in stochastic processes by combining Cantor embeddings with Wasserstein distances, enabling effective detection of predictive equivalences in symbolic time-series data.

Contribution

The paper proposes a new geometric framework using Cantor embeddings and Wasserstein distances for identifying predictive states, extending analysis to complex infinite-state processes.

Findings

Hierarchical clustering reveals process structures effectively.

The method applies to both simple and complex stochastic models.

Dimension reduction provides clear insights into temporal dynamics.

Abstract

Predictive states for stochastic processes are a nonparametric and interpretable construct with relevance across a multitude of modeling paradigms. Recent progress on the self-supervised reconstruction of predictive states from time-series data focused on the use of reproducing kernel Hilbert spaces. Here, we examine how Wasserstein distances may be used to detect predictive equivalences in symbolic data. We compute Wasserstein distances between distributions over sequences ("predictions"), using a finite-dimensional embedding of sequences based on the Cantor for the underlying geometry. We show that exploratory data analysis using the resulting geometry via hierarchical clustering and dimension reduction provides insight into the temporal structure of processes ranging from the relatively simple (e.g., finite-state hidden Markov models) to the very complex (e.g., infinite-state indexed grammars).