DDOT: A Derivative-directed Dual-decoder Ordinary Differential Equation Transformer for Dynamic System Modeling

TL;DR

This paper introduces DDOT, a transformer-based model that accurately reconstructs multidimensional ODEs from data, outperforming existing methods and demonstrating practical utility in real-world scenarios.

Contribution

The paper proposes DDOT, a novel derivative-directed dual-decoder transformer that improves symbolic ODE reconstruction and generalization, addressing limitations of previous single-trajectory approaches.

Findings

DDOT outperforms existing symbolic regression methods in ODE reconstruction.

DDOT achieves higher $P(R^2 > 0.9)$ scores in generalization tasks.

DDOT reduces divergence difference (DIV-diff) significantly.

Abstract

Uncovering the underlying ordinary differential equations (ODEs) that govern dynamic systems is crucial for advancing our understanding of complex phenomena. Traditional symbolic regression methods often struggle to capture the temporal dynamics and intervariable correlations inherent in ODEs. ODEFormer, a state-of-the-art method for inferring multidimensional ODEs from single trajectories, has made notable progress. However, its focus on single-trajectory evaluation is highly sensitive to initial starting points, which may not fully reflect true performance. To address this, we propose the divergence difference metric (DIV-diff), which evaluates divergence over a grid of points within the target region, offering a comprehensive and stable analysis of the variable space. Alongside, we introduce DDOT (Derivative-Directed Dual-Decoder Ordinary Differential Equation Transformer), a transformer-based model designed to reconstruct multidimensional ODEs in symbolic form. By incorporating an auxiliary task predicting the ODE's derivative, DDOT effectively captures both structure and dynamic behavior. Experiments on ODEBench show DDOT outperforms existing symbolic regression methods, achieving an absolute improvement of 4.58% and 1.62% in $P(R^2 > 0.9)$ for reconstruction and generalization tasks, respectively, and an absolute reduction of 3.55% in DIV-diff. Furthermore, DDOT demonstrates real-world applicability on an anesthesia dataset, highlighting its practical impact.