KAM Theory Meets Statistical Learning Theory: Hamiltonian Neural Networks with Non-Zero Training Loss

TL;DR

This paper combines statistical learning theory and KAM theory to analyze Hamiltonian neural networks with non-zero training error, providing theoretical insights into their behavior as perturbed Hamiltonian systems.

Contribution

It introduces a generalization error bound for Hamiltonian neural networks using covering number estimates, bridging neural network approximation and Hamiltonian perturbation theory.

Findings

Provides an $L^ Infty$ bound on the Hamiltonian for networks with non-zero error.

Establishes a theoretical framework connecting neural network errors with Hamiltonian perturbations.

Extends the applicability of KAM theory to neural network approximations of Hamiltonian systems.

Abstract

Many physical phenomena are described by Hamiltonian mechanics using an energy function (the Hamiltonian). Recently, the Hamiltonian neural network, which approximates the Hamiltonian as a neural network, and its extensions have attracted much attention. This is a very powerful method, but its use in theoretical studies remains limited. In this study, by combining the statistical learning theory and Kolmogorov-Arnold-Moser (KAM) theory, we provide a theoretical analysis of the behavior of Hamiltonian neural networks when the learning error is not completely zero. A Hamiltonian neural network with non-zero errors can be considered as a perturbation from the true dynamics, and the perturbation theory of the Hamilton equation is widely known as the KAM theory. To apply the KAM theory, we provide a generalization error bound for Hamiltonian neural networks by deriving an estimate of the covering number of the gradient of the multi-layer perceptron, which is the key ingredient of the model. This error bound gives an $L^\infty$ bound on the Hamiltonian that is required in the application of the KAM theory.