CoDBench: A Critical Evaluation of Data-driven Models for Continuous Dynamical Systems

TL;DR

This paper introduces CoDBench, a comprehensive benchmarking suite evaluating various data-driven models for continuous dynamical systems across multiple datasets, highlighting current limitations and guiding future research.

Contribution

The paper presents CoDBench, the first extensive benchmark comparing 11 state-of-the-art models across diverse dynamical systems datasets, revealing strengths and weaknesses of current approaches.

Findings

Current models struggle with newer mechanics datasets.

Data efficiency varies significantly across models.

Robustness to noise remains a challenge for many models.

Abstract

Continuous dynamical systems, characterized by differential equations, are ubiquitously used to model several important problems: plasma dynamics, flow through porous media, weather forecasting, and epidemic dynamics. Recently, a wide range of data-driven models has been used successfully to model these systems. However, in contrast to established fields like computer vision, limited studies are available analyzing the strengths and potential applications of different classes of these models that could steer decision-making in scientific machine learning. Here, we introduce CodBench, an exhaustive benchmarking suite comprising 11 state-of-the-art data-driven models for solving differential equations. Specifically, we comprehensively evaluate 4 distinct categories of models, viz., feed forward neural networks, deep operator regression models, frequency-based neural operators, and transformer architectures against 8 widely applicable benchmark datasets encompassing challenges from fluid and solid mechanics. We conduct extensive experiments, assessing the operators' capabilities in learning, zero-shot super-resolution, data efficiency, robustness to noise, and computational efficiency. Interestingly, our findings highlight that current operators struggle with the newer mechanics datasets, motivating the need for more robust neural operators. All the datasets and codes will be shared in an easy-to-use fashion for the scientific community. We hope this resource will be an impetus for accelerated progress and exploration in modeling dynamical systems.