An Efficient Finite Difference Method for Parameter Sensitivities of Continuous Time Markov Chains

TL;DR

This paper introduces a coupling-based finite difference method for efficiently computing parameter sensitivities in continuous time Markov chains, significantly reducing estimator variance in applications like biochemical kinetics.

Contribution

The paper presents a novel coupling approach that lowers variance in sensitivity estimations for continuous time Markov chains, applicable to biochemical and other stochastic models.

Findings

Variance of the estimator is reduced by an order of magnitude.

The method is as easy to implement as standard algorithms.

Significant variance reduction compared to existing methods.

Abstract

We present an efficient finite difference method for the computation of parameter sensitivities that is applicable to a wide class of continuous time Markov chain models. The estimator for the method is constructed by coupling the perturbed and nominal processes in a natural manner, and the analysis proceeds by utilizing a martingale representation for the coupled processes. The variance of the resulting estimator is shown to be an order of magnitude lower due to the coupling. We conclude that the proposed method produces an estimator with a lower variance than other methods, including the use of Common Random Numbers, in most situations. Often the variance reduction is substantial. The method is no harder to implement than any standard continuous time Markov chain algorithm, such as "Gillespie's algorithm." The motivating class of models, and the source of our examples, are the stochastic chemical kinetic models commonly used in the biosciences, though other natural application areas include population processes and queuing networks.