Informational and Causal Architecture of Continuous-time Renewal and Hidden Semi-Markov Processes

TL;DR

This paper develops minimal predictive models ({ extepsilon}-machines) for continuous-time renewal and hidden semi-Markov processes, providing analytical expressions for their informational properties and extending the framework to complex process classes.

Contribution

It introduces a unified framework for modeling continuous-time processes with hybrid or continuous causal states, including explicit formulas for key information measures.

Findings

Derived closed-form expressions for statistical complexities and excess entropies.

Analyzed the causal structure of renewal and semi-Markov processes.

Extended the models to unifilar hidden semi-Markov processes.

Abstract

We introduce the minimal maximally predictive models ({\epsilon}-machines) of processes generated by certain hidden semi-Markov models. Their causal states are either hybrid discrete-continuous or continuous random variables and causal-state transitions are described by partial differential equations. Closed-form expressions are given for statistical complexities, excess entropies, and differential information anatomy rates. We present a complete analysis of the {\epsilon}-machines of continuous-time renewal processes and, then, extend this to processes generated by unifilar hidden semi-Markov models and semi-Markov models. Our information-theoretic analysis leads to new expressions for the entropy rate and the rates of related information measures for these very general continuous-time process classes.