Physics-Modeled Neural Networks

TL;DR

DynPMNNs are a new class of neural networks that model hidden layers as solutions to differential equations, integrating physical models and dynamical systems into deep learning.

Contribution

We introduce DynPMNNs, a physics-inspired continuous-time neural network framework grounded in RKBSs, with a concrete FitzHugh--Nagumo implementation and competitive experimental results.

Findings

DynPMNNs achieve competitive performance with fewer parameters.

The framework connects dynamical systems theory with deep learning.

Experimental results on California Housing dataset demonstrate effectiveness.

Abstract

We introduce \emph{Dynamical Physics-Modeled Neural Networks} (DynPMNNs), a continuous-time deep learning architecture in which each hidden layer is defined as the solution of an ordinary differential equation. Unlike classical feed-forward networks, this approach replaces static activation functions with time-evolving dynamical systems, providing a biologically inspired interpretation of hidden-layer behavior and enabling the integration of physically meaningful models. The framework is rigorously grounded in Reproducing Kernel Banach Spaces (RKBSs), allowing DynPMNNs to be characterized as finite-dimensional solutions of an abstract training problem and revealing structural connections with standard neural networks. We present a concrete implementation based on the FitzHugh--Nagumo model for neuronal activation, where numerical ODE solvers are embedded into the computational graph via Euler-type schemes. Both network weights and dynamical parameters are trained jointly. Through experiments on the California Housing dataset, we compare DynPMNNs with Neural ODEs (NODEs) and Closed-form Continuous-Time Networks (CfCs). Despite using fewer trainable parameters, DynPMNNs achieve competitive performance. These results position DynPMNNs as a principled bridge between dynamical systems and deep learning, with promising directions for further research in expressivity, stability, and physics-based modeling.