Data-driven functional state estimation of complex networks

TL;DR

This paper presents a data-driven approach for estimating specific internal states of complex networks without requiring detailed system models, enabling real-time monitoring and control.

Contribution

It introduces a novel functional observability framework and methods to construct observers directly from data, applicable even to unobservable and nonlinear systems.

Findings

Achieved effective state estimation in water networks and power grids.

Demonstrated robustness to noise and applicability to nonlinear systems.

Provided a practical tool for real-time monitoring without detailed models.

Abstract

The internal state of a dynamical system, a set of variables that defines its evolving configuration, is often hidden and cannot be fully measured, posing a central challenge for real-time monitoring and control. While observers are designed to estimate these latent states from sensor outputs, their classical designs rely on precise system models, which are often unattainable for complex network systems. Here, we introduce a data-driven framework for estimating a targeted set of state variables, known as functional observers, without identifying the model parameters. We establish a fundamental functional observability criterion based on historical trajectories that guarantees the existence of such observers. We then develop methods to construct observers using either input-output data or partial state data. These observers match or exceed the performance of model-based counterparts while remaining applicable even to unobservable systems. The framework incorporates noise mitigation and can be easily extended to nonlinear networks via Koopman embeddings. We demonstrate its broad utility through applications including sensor fault detection in water networks, load-frequency control in power grids, and target estimation in nonlinear neuronal systems. Our work provides a practical route for real-time target state inference in complex systems where models are unavailable.