Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEs

TL;DR

Hyperfastrl introduces a hypernetwork-based reinforcement learning approach that enables unified control of parametric chaotic PDEs, improving robustness and efficiency across varying parameters.

Contribution

The paper develops Hyperfastrl, a novel hypernetwork reinforcement learning framework that maps physical parameters directly to control policies, reducing the need for regime-specific tuning.

Findings

All tested hypernetwork forms achieved robust stabilization.

Kolmogorov-Arnold networks provided the best extrapolation performance.

Parallelization reduced training time by 37% with minimal reward loss.

Abstract

Spatiotemporal chaos in fluid systems exhibits severe parametric sensitivity, rendering classical adjoint-based optimal control intractable because each operating regime requires recomputing the control law. We address this bottleneck with hyperFastRL, a parameter-conditioned reinforcement learning framework that leverages Hypernetworks to shift from tuning isolated controllers per-regime to learning a unified parametric control manifold. By mapping a physical forcing parameter {\mu} directly to the weights of a spatial feedback policy, the architecture cleanly decouples parametric adaptation from spatial boundary stabilization. To overcome the extreme variance inherent to chaotic reward landscapes, we deploy a pessimistic distributional value estimation over a massively parallel environment ensemble. We evaluate three Hypernetwork functional forms, ranging from residual MLPs to periodic Fourier and Kolmogorov-Arnold (KAN) representations, on the Kuramoto-Sivashinsky equation under varying spatial forcing. All forms achieve robust stabilization. KAN yields the most consistent energy-cascade suppression and tracking across unseen parametrizations, while Fourier networks exhibit worse extrapolation variability. Furthermore, leveraging high-throughput parallelization allows us to intentionally trade a fraction of peak asymptotic reward for a 37% reduction in training wall-clock time, identifying an optimal operating regime for practical deployment in complex, parameter-varying chaotic PDEs.