Resolving gradient pathology in physics-informed epidemiological models

TL;DR

This paper introduces conflict-gated gradient scaling (CGGS), a novel method to stabilize and accelerate physics-informed neural network training in epidemiology by dynamically managing gradient conflicts, leading to better convergence and parameter estimation.

Contribution

We propose CGGS, a new gradient conflict resolution technique that preserves convergence guarantees and improves training stability and efficiency in physics-informed epidemiological models.

Findings

CGGS improves convergence speed over baseline methods.

It enhances parameter estimation accuracy in stiff epidemiological systems.

The method induces a curriculum learning effect, aiding training.

Abstract

Physics-informed neural networks (PINNs) are increasingly used in mathematical epidemiology to bridge the gap between noisy clinical data and compartmental models, such as the susceptible-exposed-infected-removed (SEIR) model. However, training these hybrid networks is often unstable due to competing optimization objectives. As established in recent literature on ``gradient pathology," the gradient vectors derived from the data loss and the physical residual often point in conflicting directions, leading to slow convergence or optimization deadlock. While existing methods attempt to resolve this by balancing gradient magnitudes or projecting conflicting vectors, we propose a novel method, conflict-gated gradient scaling (CGGS), to address gradient conflicts in physics-informed neural networks for epidemiological modelling, ensuring stable and efficient training and a computationally efficient alternative. This method utilizes the cosine similarity between the data and physics gradients to dynamically modulate the penalty weight. Unlike standard annealing schemes that only normalize scales, CGGS acts as a geometric gate: it suppresses the physical constraint when directional conflict is high, allowing the optimizer to prioritize data fidelity, and restores the constraint when gradients align. We prove that this gating mechanism preserves the standard $O(1/T)$ convergence rate for smooth non-convex objectives, a guarantee that fails under fixed-weight or magnitude-balanced training when gradients conflict. We demonstrate that this mechanism autonomously induces a curriculum learning effect, improving parameter estimation in stiff epidemiological systems compared to magnitude-based baselines. Our empirical results show improved peak recovery and convergence over magnitude-based methods.