Markov matrix perturbations to optimize dynamical and entropy functionals

TL;DR

This paper develops perturbation-based linear algorithms to optimize entropy-related functionals in Markov matrices, enabling data-driven control of dynamical systems without explicit equations.

Contribution

It introduces a novel perturbation theory approach for Markov matrices that facilitates the optimization of entropy and related functionals in complex dynamical systems.

Findings

Algorithms successfully optimize entropy and divergence measures.

Applicable to Markov chains from transfer operators and turbulent flow models.

Provides a data-driven method to interpret perturbations physically.

Abstract

An important problem in applied dynamical systems is to compute the external forcing that provokes the largest response of a desired observable quantity. For this, we investigate the perturbation theory of Markov matrices in connection with linear response theory in statistical physics. We use perturbative expansions to derive linear algorithms to optimize physically relevant quantities such as: entropy, Kullback-Liebler-divergence and entropy production of Markov matrices and their related probability vectors. These optimization algorithms are applied to Markov chain representations of discrete and continuous flows in and out of equilibrium. We consider Markov matrix representations originating from Ulam-type approximations of transfer operators and a reduced order model of a turbulent flow based on unstable periodic orbits theory. We also propose a numerical protocol to recast matrix perturbations into vector field perturbations. The results allow to physically interpret the obtained optimizing perturbations without knowledge of the underlying equations, in a data-driven way.