Variational analysis of inference from dynamical systems

TL;DR

This paper develops a variational framework to analyze empirical risk-based inference in dynamical systems, providing insights into the convergence, identifiability, and robustness of parameter estimates.

Contribution

It introduces a novel variational approach to characterize the asymptotic behavior of inference procedures for dynamical systems, including system identification from noisy, quantized data.

Findings

Empirical risk converges to a constant expressed via minimal expected loss over invariant couplings.

The set of minimizing joinings is convex, compact, and characterizes asymptotic parameter behavior.

Application to maximum likelihood, nonlinear regression, and noisy system identification demonstrates the framework's versatility.

Abstract

We introduce and study a variational framework for the analysis of empirical risk based inference for dynamical systems and ergodic processes. The analysis applies to a two-stage estimation procedure in which (i) the trajectory of an observed (but unknown) system is fit to a trajectory from a known reference system by minimizing cumulative per-state loss, and (ii) a parameter estimate is obtained from the initial state of the best fit reference trajectory. We show that the empirical risk of the best fit trajectory converges almost surely to a constant that can be expressed in a variational form as the minimal expected loss over dynamically invariant couplings (joinings) of the observed and reference systems. Moreover, we establish that the family of joinings minimizing the expected loss is convex and compact, and that it fully characterizes the asymptotic behavior of the estimated parameters, addressing both identifiability and misspecification. The two-stage estimation framework and associated variational analysis apply to a broad family of empirical risk miminization procedures for dependent observations. To illustrate this, we apply variational analysis to the well studied problems of maximum likelihood and non-linear regression, and then undertake an extended analysis of system identification from quantized trajectories subject to noise, a problem of interest in dynamics, where the models themselves exhibit dynamical behavior across time.