Binned Spectral Power Loss for Improved Prediction of Chaotic Systems

TL;DR

This paper introduces the Binned Spectral Power (BSP) Loss, a frequency-domain loss function that reduces spectral bias in deep learning models, improving long-term predictions of chaotic systems like turbulent flows.

Contribution

The novel BSP loss explicitly penalizes spectral deviations, enhancing stability and spectral accuracy in neural network forecasts without changing model architecture.

Findings

BSP loss reduces spectral bias in chaotic system predictions.

Models trained with BSP loss show improved stability in long-term forecasts.

Enhanced spectral fidelity achieved without architectural modifications.

Abstract

Forecasting multiscale chaotic dynamical systems, such as turbulent flows, with deep learning remains a formidable challenge due to the spectral bias of neural networks, which hinders the accurate representation of fine-scale structures in long-term predictions. This issue is exacerbated when models are deployed autoregressively, leading to compounding errors and instability. In this work, we introduce a novel approach to mitigate the spectral bias, which we call the Binned Spectral Power (BSP) Loss. The BSP loss is a frequency-domain loss function that adaptively weighs errors in predicting both larger and smaller scales of the dataset. Unlike traditional losses that focus on pointwise misfits, our BSP loss explicitly penalizes deviations in the energy distribution across different scales, promoting stable and physically consistent predictions. We demonstrate that the BSP loss mitigates the well-known problem of spectral bias in deep learning. We further validate our approach for the data-driven high-dimensional time-series forecasting of a range of benchmark chaotic systems, which are typically intractable due to spectral bias, culminating in experiments on canonical turbulent flow benchmarks. Our results demonstrate that the BSP loss significantly improves the stability and spectral accuracy of neural forecasting models without requiring architectural modifications. By directly targeting spectral consistency, our approach paves the way for more robust deep learning models for long-term forecasting of chaotic dynamical systems.