Neural Context Flows for Meta-Learning of Dynamical Systems

TL;DR

Neural Context Flow (NCF) is a meta-learning framework that enhances Neural ODEs' ability to adapt to unseen dynamic behaviors, especially with unobserved parameters, by using Taylor expansion-based contextual self-modulation and uncertainty estimation.

Contribution

This paper introduces NCF, a novel meta-learning approach with theoretical guarantees that improves NODEs' out-of-distribution performance and interpretability in dynamical systems.

Findings

NCF achieves state-of-the-art OOD performance on 5 out of 6 benchmarks.

The framework effectively models unobserved parameter variations.

Extensive experiments reveal the interpretability and flexibility of NCF.

Abstract

Neural Ordinary Differential Equations (NODEs) often struggle to adapt to new dynamic behaviors caused by parameter changes in the underlying physical system, even when these dynamics are similar to previously observed behaviors. This problem becomes more challenging when the changing parameters are unobserved, meaning their value or influence cannot be directly measured when collecting data. To address this issue, we introduce Neural Context Flow (NCF), a robust and interpretable Meta-Learning framework that includes uncertainty estimation. NCF uses Taylor expansion to enable contextual self-modulation, allowing context vectors to influence dynamics from other domains while also modulating themselves. After establishing theoretical guarantees, we empirically test NCF and compare it to related adaptation methods. Our results show that NCF achieves state-of-the-art Out-of-Distribution performance on 5 out of 6 linear and non-linear benchmark problems. Through extensive experiments, we explore the flexible model architecture of NCF and the encoded representations within the learned context vectors. Our findings highlight the potential implications of NCF for foundational models in the physical sciences, offering a promising approach to improving the adaptability and generalization of NODEs in various scientific applications. Our code is openly available at https://github.com/ddrous/ncflow.