Neural Network Perturbation Theory (NNPT): Learning Residual Corrections from Exact Solutions

TL;DR

Neural Network Perturbation Theory (NNPT) enables neural networks to learn residual corrections from known solutions, significantly reducing errors and revealing capacity thresholds linked to physical chaos in complex systems.

Contribution

This work introduces NNPT, a novel framework where neural networks learn perturbative residuals instead of complete solutions, improving efficiency and uncovering capacity transitions related to system chaos.

Findings

Correction learning reduces validation error by 28-54x.

Capacity jumps sharply at chaos transition point.

Symplectic integrator maintains high energy conservation.

Abstract

Many complex physical systems admit natural decomposition into an exactly solvable component and a perturbative correction. Rather than training neural networks to learn complete trajectories from scratch, we introduce Neural Network Perturbation Theory (NNPT), where networks predict only residual perturbations after analytically subtracting known exact solutions. We validate this framework through systematic comparison: using identical 2x32 architectures, correction learning achieves 28-54x lower validation error compared to networks trained on complete trajectories. Using the gravitational three-body problem as a test bed, we investigate capacity transitions in fixed-architecture multilayer perceptrons as Jovian mass varies from 0.05 to 30 times its physical value. An equalized-accuracy protocol reveals that both minimal network capacity and training time exhibit sharp transitions at f_c = 15.6+-1.0, where the system enters a strongly chaotic regime. At this transition, minimal capacity jumps approximately sevenfold from ~1,200 to ~8,600 parameters (architectures 2x32 and 3x64). Preliminary exploration of sequential two-stage corrections suggests that first-stage networks already capture dominant perturbative features. Our symplectic integrator maintains relative energy conservation below 2x10^-7 throughout, confirming that transitions reflect physical complexity rather than numerical error. Our results establish correction learning as a general strategy for parameter-efficient surrogates and demonstrate that physical complexity imposes fundamental capacity barriers on fixed-architecture networks at chaos onset.