Information Theoretic Limits on Learning Stochastic Differential Equations

TL;DR

This paper establishes fundamental limits on the time required to learn the parameters of stochastic differential equations from observed data, using information-theoretic methods, applicable to linear and nonlinear cases.

Contribution

It introduces a general lower bound on observation time for learning SDE parameters, connecting information theory with stochastic process estimation.

Findings

Lower bounds match upper bounds for certain classes of SDEs

Quantitative estimates for learning sparse and dense matrix interactions

Applicable to both linear and nonlinear stochastic differential equations

Abstract

Consider the problem of learning the drift coefficient of a stochastic differential equation from a sample path. In this paper, we assume that the drift is parametrized by a high dimensional vector. We address the question of how long the system needs to be observed in order to learn this vector of parameters. We prove a general lower bound on this time complexity by using a characterization of mutual information as time integral of conditional variance, due to Kadota, Zakai, and Ziv. This general lower bound is applied to specific classes of linear and non-linear stochastic differential equations. In the linear case, the problem under consideration is the one of learning a matrix of interaction coefficients. We evaluate our lower bound for ensembles of sparse and dense random matrices. The resulting estimates match the qualitative behavior of upper bounds achieved by computationally efficient procedures.