Training Hamiltonian neural networks without backpropagation

TL;DR

This paper introduces a novel backpropagation-free algorithm for training Hamiltonian neural networks, significantly improving training speed and accuracy in modeling dynamical systems.

Contribution

The authors develop a data-driven, backpropagation-free method that outperforms traditional gradient-based training for Hamiltonian neural networks.

Findings

Data-driven sampling outperforms data-agnostic sampling.

Our method is over 100 times faster on CPUs.

Achieves four orders of magnitude higher accuracy in chaotic systems.

Abstract

Neural networks that synergistically integrate data and physical laws offer great promise in modeling dynamical systems. However, iterative gradient-based optimization of network parameters is often computationally expensive and suffers from slow convergence. In this work, we present a backpropagation-free algorithm to accelerate the training of neural networks for approximating Hamiltonian systems through data-agnostic and data-driven algorithms. We empirically show that data-driven sampling of the network parameters outperforms data-agnostic sampling or the traditional gradient-based iterative optimization of the network parameters when approximating functions with steep gradients or wide input domains. We demonstrate that our approach is more than 100 times faster with CPUs than the traditionally trained Hamiltonian Neural Networks using gradient-based iterative optimization and is more than four orders of magnitude accurate in chaotic examples, including the H\'enon-Heiles system.