Unifying physical systems' inductive biases in neural ODE using dynamics constraints

TL;DR

This paper introduces a unifying regularization-based approach for neural ODEs that incorporates physical inductive biases, enabling modeling of both energy-conserving and dissipative systems without altering neural network architectures.

Contribution

A simple, architecture-agnostic regularization method that generalizes energy conservation principles to various physical systems in neural ODE modeling.

Findings

Applicable to both conservative and dissipative systems

Does not require changes to neural network architecture

Facilitates validation of new physical inductive biases

Abstract

Conservation of energy is at the core of many physical phenomena and dynamical systems. There have been a significant number of works in the past few years aimed at predicting the trajectory of motion of dynamical systems using neural networks while adhering to the law of conservation of energy. Most of these works are inspired by classical mechanics such as Hamiltonian and Lagrangian mechanics as well as Neural Ordinary Differential Equations. While these works have been shown to work well in specific domains respectively, there is a lack of a unifying method that is more generally applicable without requiring significant changes to the neural network architectures. In this work, we aim to address this issue by providing a simple method that could be applied to not just energy-conserving systems, but also dissipative systems, by including a different inductive bias in different cases in the form of a regularisation term in the loss function. The proposed method does not require changing the neural network architecture and could form the basis to validate a novel idea, therefore showing promises to accelerate research in this direction.