FANoS-v2: Feedback-Controlled Momentum with Thermostat Damping for Lightweight Neural Optimization

TL;DR

FANoS-v2 is a PyTorch optimizer that combines feedback-controlled momentum with thermostat damping, offering improved training performance at the cost of increased computation time, and is presented as an alpha-stage research tool.

Contribution

The paper provides a complete mathematical specification of FANoS-v2, including novel control mechanisms and diagnostics, and reports experimental results demonstrating its performance.

Findings

FANoS-v2 achieves mean top-1 gains over AdamW on several datasets.

It incurs approximately 50-60% higher wall-clock time.

Preliminary tests show mixed results in scientific and EEG applications.

Abstract

\FANOS{} is a PyTorch optimizer that augments RMS-preconditioned momentum with a scalar feedback controller over update energy. The public reference implementation stores momentum in parameter-update units, applies a non-negative thermostat damping coefficient, supports diagonal, factored, and raw-gradient preconditioning, and exposes diagnostics intended for stability audits. This study gives a complete mathematical specification of the released optimizer, including the exact parameter-unit update, the study-equation physical update mode, bounded log-ratio thermostat control, adaptive preconditioner softening, warmup guardrails, and the experimental \Fast{} profile. We report the v0.2 evidence: five-seed reduced-sample MNIST, Fashion-MNIST, and CIFAR-10 experiments show mean top-1 gains of 0.889, 2.197, and 2.666 percentage points over AdamW for \Fast{}, but with 49.8\%, 61.6\%, and 56.8\% higher wall-clock time. Preliminary scientific, PINN, and EEG smoke tests are mixed and are treated as hypothesis-generating only. The evidence supports \FANOS{} as an alpha-stage research optimizer with a reproducible lightweight-vision signal and an explicit runtime bottleneck.