Sampling and Filtering with Markov Chains

TL;DR

This paper introduces a new framework for sampling and filtering in continuous-time Markov chains, providing novel algorithms and equations applicable to financial models and Hidden Markov Models, enhancing simulation and estimation techniques.

Contribution

It develops a rate change formula and new filtering algorithms for continuous-time Markov chains, extending to Hidden Markov Models with broad applications.

Findings

Derived a Markov chain importance sampling method

Formulated filtering equations similar to classical stochastic equations

Applied results to financial models and disease progression tracking

Abstract

A continuous-time Markov chain rate change formula for simulation, model selection, filtering and theory is proven. It is used to develop Markov chain importance sampling, rejection sampling, branching particle filtering algorithms and filtering equations akin to the Duncan-Mortensen-Zakai equation and the Fujisaki-Kallianpur-Kunita equation but for Markov signals with general continuous-time Markov chain observations. A direct method of solving these filtering equations is given that, for example, applies to trend, volatility and/or parameter estimation in financial models given tick-by-tick market data. All the results also apply to continuous-time Hidden Markov Models (CTHMM), which have become important in applications like disease progression tracking, as special cases and the corresponding CTHMM results are stated as corollaries.