Information entropy and temperature of the binary Markov chains

TL;DR

This paper introduces two methods to define the information temperature of binary N-th order Markov chains, comparing them with Ising models and analyzing entropy derivatives, with applications to literary texts.

Contribution

It presents novel approaches for calculating the information temperature of Markov chains, including a comparison with Ising models and analysis of weakly correlated chains.

Findings

Both approaches yield similar results for nearest-neighbor interactions.

The Markov-Ising correspondence becomes cumbersome for N>3.

Application to literary texts demonstrates practical utility.

Abstract

We propose two different approaches for introducing the information temperature of the binary N-th order Markov chains. The first approach is based on comparing the Markov sequences with the equilibrium Ising chains at given temperatures. The second approach uses probabilities of finite-length subsequences of symbols occurring, which determine their entropies. The derivative of the entropy with respect to the energy gives the information temperature measured on the scale of introduced energy. For the case of nearest-neighbor spin/symbol interaction, both approaches provide similar results. However, the method based on the correspondence of the N-step Markov and Ising chains appears to be very cumbersome for N>3. We also introduce the information temperature for the weakly correlated one-parametric Markov chains and present results for the step-wise and power memory functions. An application of the developed method to obtain the information temperature of some literary texts is given.