Robust Learning of Heterogeneous Dynamic Systems

TL;DR

This paper introduces a distributionally robust learning method for modeling heterogeneous dynamic systems using ODEs, with theoretical guarantees and improved generalization demonstrated through simulations and EEG data analysis.

Contribution

It proposes a novel robust estimation approach for heterogeneous ODE systems, including a bi-level stabilization and theoretical analysis of its properties.

Findings

Improved generalization performance over existing methods.

The estimator admits an explicit weighted average representation.

Theoretical guarantees include consistency and error bounds.

Abstract

Ordinary differential equations (ODEs) provide a powerful framework for modeling dynamic systems arising in a wide range of scientific domains. However, most existing ODE methods focus on a single system, and do not adequately address the problem of learning shared patterns from multiple heterogeneous dynamic systems. In this article, we propose a novel distributionally robust learning approach for modeling heterogeneous ODE systems. Specifically, we construct a robust dynamic system by maximizing a worst-case reward over an uncertainty class formed by convex combinations of the derivatives of trajectories. We show the resulting estimator admits an explicit weighted average representation, where the weights are obtained from a quadratic optimization that balances information across multiple data sources. We further develop a bi-level stabilization procedure to address potential instability in estimation. We establish rigorous theoretical guarantees for the proposed method, including consistency of the stabilized weights, error bound for robust trajectory estimation, and asymptotical validity of pointwise confidence interval. We demonstrate that the proposed method considerably improves the generalization performance compared to the alternative solutions through both extensive simulations and the analysis of an intracranial electroencephalogram data.