Learning System Dynamics without Forgetting
Xikun Zhang, Dongjin Song, Yushan Jiang, Yixin Chen, Dacheng Tao
TL;DR
This paper introduces a new continual learning framework for systems with evolving dynamics, proposing the MS-GODE model and a biological systems benchmark to improve trajectory prediction across changing environments.
Contribution
It presents the Mode-switching Graph ODE (MS-GODE) model for continual dynamics learning and introduces Bio-CDL, a benchmark for biological dynamic systems.
Findings
MS-GODE effectively learns across varying dynamics.
Bio-CDL benchmark enriches research in dynamic systems.
Proposed method outperforms existing approaches.
Abstract
Observation-based trajectory prediction for systems with unknown dynamics is essential in fields such as physics and biology. Most existing approaches are limited to learning within a single system with fixed dynamics patterns. However, many real-world applications require learning across systems with evolving dynamics patterns, a challenge that has been largely overlooked. To address this, we systematically investigate the problem of Continual Dynamics Learning (CDL), examining task configurations and evaluating the applicability of existing techniques, while identifying key challenges. In response, we propose the Mode-switching Graph ODE (MS-GODE) model, which integrates the strengths LG-ODE and sub-network learning with a mode-switching module, enabling efficient learning over varying dynamics. Moreover, we construct a novel benchmark of biological dynamic systems for CDL, Bio-CDL,…
Peer Reviews
Decision·ICLR 2025 Poster
1. The paper studies an interesting problem in dynamical system modeling under distribution shifts. Though I feel the motivation example can be changed to more realistic one (see below). 2. The paper proposes a useful benchmark for evaluating the distribution shifts in dynamical system modeling, on biological cellular systems. 3. The proposed method is able to achieve good performance over selected baselines.
1. While I in general get the motivation of this paper, I feel the examples used in the introduction section need to be further improved. First for Figure 1, there are so many contents/fonts in the figure that are not well-explained in the caption or the main text. It is suggested to use for example legend to denote different kinetic factors, using some boxes to denote the overall system consists of multiple objects. Here for me, it is hard to read from this figure along with its current explana
- Continual learning setup on interacting system dynamics is a novel problem. Since very recent works have explored evolving system dynamics, continual learning can provide a new insight on the related research community. - As the authors mentioned in L131, neural ODE has been focused on a single dynamical systems. Combination of neural ODE with masked networks is a new approach. - The paper introduces a novel benchmark using biological cellular systems. Considering many benchmarks on interac
- While I still consider the combination of neural ODE with continual learning interesting, technical novelties seem somewhat limited. For example, both the network design (e.g., LG-ODE, NRI) and learning method (e.g., masked-based CL, edge-popup algorithm) are already well known. I think, unique technical contributions this paper had made should have been clearly presented. - Also, the motivation of CDL needs to be better clarified. I don’t think continual learning is always required for all e
The paper's primary innovation lies in recognizing and formalizing the CDL problem, which has been overlooked despite its practical importance. The MS-GODE architecture effectively combines proven techniques (Graph ODEs, sub-network learning) with a novel mode-switching mechanism, demonstrating strong performance across different system configurations. The introduction of Bio-CDL, a benchmark featuring biological cellular systems, significantly enriches the field beyond traditional physics simul
- Several important baselines are notably absent from the comparison, including CG-ODE, GG-ODE, PG-ODE, and HOPE. - The experimental validation would be more convincing if it included tests on human motion and molecular dynamics (MD17) datasets, which could reveal how the approach handles different types of dynamic patterns. The model shows sensitivity to hyperparameters, particularly dropout rates and mask selection strategies, but lacks clear guidelines for parameter selection in practical a
Code & Models
Videos
Taxonomy
TopicsNeural Networks and Applications
MethodsFocus