Characterizing control between interacting subsystems with deep Jacobian estimation

TL;DR

This paper introduces a deep learning framework called JacobianODE for estimating Jacobians from time-series data to characterize nonlinear control interactions between subsystems, with applications to neural networks.

Contribution

It develops a novel data-driven nonlinear control-theoretic method to infer subsystem interactions via Jacobian estimation, outperforming existing methods on complex systems.

Findings

JacobianODE outperforms existing Jacobian estimation methods.

Applied to RNNs, it reveals increased control of sensory over cognitive areas during learning.

Enables direct manipulation of neural network behavior through Jacobian-based control.

Abstract

Biological function arises through the dynamical interactions of multiple subsystems, including those between brain areas, within gene regulatory networks, and more. A common approach to understanding these systems is to model the dynamics of each subsystem and characterize communication between them. An alternative approach is through the lens of control theory: how the subsystems control one another. This approach involves inferring the directionality, strength, and contextual modulation of control between subsystems. However, methods for understanding subsystem control are typically linear and cannot adequately describe the rich contextual effects enabled by nonlinear complex systems. To bridge this gap, we devise a data-driven nonlinear control-theoretic framework to characterize subsystem interactions via the Jacobian of the dynamics. We address the challenge of learning Jacobians from time-series data by proposing the JacobianODE, a deep learning method that leverages properties of the Jacobian to directly estimate it for arbitrary dynamical systems from data alone. We show that JacobianODEs outperform existing Jacobian estimation methods on challenging systems, including high-dimensional chaos. Applying our approach to a multi-area recurrent neural network (RNN) trained on a working memory selection task, we show that the "sensory" area gains greater control over the "cognitive" area over learning. Furthermore, we leverage the JacobianODE to directly control the trained RNN, enabling precise manipulation of its behavior. Our work lays the foundation for a theoretically grounded and data-driven understanding of interactions among biological subsystems.