When do neural ordinary differential equations generalize on complex networks?

TL;DR

This paper investigates the generalization capabilities of neural ODEs on complex, graph-structured data, revealing that degree heterogeneity and dynamical system type are key factors influencing their performance across different graph sizes and structures.

Contribution

It provides the first systematic analysis of neural ODEs' ability to generalize on complex networks with varying structures, highlighting the impact of degree heterogeneity and graph properties.

Findings

Degree heterogeneity significantly affects neural ODE generalization.

Type of dynamical system influences the ability to generalize across graphs.

Average clustering has a secondary impact on performance.

Abstract

Neural ordinary differential equations (neural ODEs) can effectively learn dynamical systems from time series data, but their behavior on graph-structured data remains poorly understood, especially when applied to graphs with different size or structure than encountered during training. We study neural ODEs ($\mathtt{nODE}$s) with vector fields following the Barab\'asi-Barzel form, trained on synthetic data from five common dynamical systems on graphs. Using the $\mathbb{S}^1$-model to generate graphs with realistic and tunable structure, we find that degree heterogeneity and the type of dynamical system are the primary factors in determining $\mathtt{nODE}$s' ability to generalize across graph sizes and properties. This extends to $\mathtt{nODE}$s' ability to capture fixed points and maintain performance amid missing data. Average clustering plays a secondary role in determining $\mathtt{nODE}$ performance. Our findings highlight $\mathtt{nODE}$s as a powerful approach to understanding complex systems but underscore challenges emerging from degree heterogeneity and clustering in realistic graphs.