A Note on Exact State Visit Probabilities in Two-State Markov Chains

TL;DR

This paper derives exact formulas for the probability of a specific state being visited exactly k times in a two-state Markov chain, correcting previous incomplete results and providing a rigorous framework validated by simulations.

Contribution

It provides a closed-form solution for state visit probabilities in two-state Markov chains, extending and correcting prior incomplete results.

Findings

Derived exact probability formulas for state visits

Validated formulas through numerical simulations

Extended previous incomplete results in Markov chain analysis

Abstract

In this note we derive the exact probability that a specific state in a two-state Markov chain is visited exactly $k$ times after $N$ transitions. We provide a closed-form solution for $\mathbb{P}(N_l = k \mid N)$, considering initial state probabilities and transition dynamics. The solution corrects and extends prior incomplete results, offering a rigorous framework for enumerating state transitions. Numerical simulations validate the derived expressions, demonstrating their applicability in stochastic modeling.