Physics-Informed Neural Networks: A Didactic Derivation of the Complete Training Cycle

TL;DR

This paper provides a detailed, step-by-step derivation of the complete training cycle for Physics-Informed Neural Networks (PINNs), including explicit numerical examples and formulas, to clarify the underlying algebra and connect to automatic differentiation.

Contribution

It offers a didactic, transparent derivation of PINN training, including explicit calculations, recursive formulas for gradients, and validation against automatic differentiation, enhancing understanding and reproducibility.

Findings

Achieved a relative L2 error of 4.290e-4 on the example problem.

Derived general recursive formulas for gradient computation in deep networks.

Provided a reproducible Jupyter/PyTorch notebook for manual and automatic gradient validation.

Abstract

This paper is a step-by-step, self-contained guide to the complete training cycle of a Physics-Informed Neural Network (PINN) -- a topic that existing tutorials and guides typically delegate to automatic differentiation libraries without exposing the underlying algebra. Using a first-order initial value problem with a known analytical solution as a running example, we walk through every stage of the process: forward propagation of both the network output and its temporal derivative, evaluation of a composite loss function built from the ODE residual and the initial condition, backpropagation of gradients -- with particular attention to the product rule that arises in hidden layers -- and a gradient descent parameter update. Every calculation is presented with explicit, verifiable numerical values using a 1-3-3-1 multilayer perceptron with two hidden layers and 22 trainable parameters. From these concrete examples, we derive general recursive formulas -- expressed as sensitivity propagation relations -- that extend the gradient computation to networks of arbitrary depth, and we connect these formulas to the automatic differentiation engines used in practice. The trained network is then validated against the exact solution, achieving a relative $L^2$ error of $4.290 \times 10^{-4}$ using only the physics-informed loss, without any data from the true solution. A companion Jupyter/PyTorch notebook reproduces every manual calculation and the full training pipeline, providing mutual validation between hand-derived and machine-computed gradients.