Spectrum-Informed Multistage Neural Networks: Multiscale Function Approximators of Machine Precision

TL;DR

This paper introduces a multistage neural network method with spectrum-informed initialization that effectively captures high-frequency features, enabling high-precision function approximation in scientific applications like turbulent flow modeling.

Contribution

The authors propose a novel multistage neural network approach leveraging spectral biases to achieve machine-precision accuracy in complex, multi-scale scientific problems.

Findings

Achieves function approximation at machine precision ($O(10^{-16})$).

Effectively captures high-frequency components in complex systems.

Addresses spectral bias in neural networks for scientific modeling.

Abstract

Deep learning frameworks have become powerful tools for approaching scientific problems such as turbulent flow, which has wide-ranging applications. In practice, however, existing scientific machine learning approaches have difficulty fitting complex, multi-scale dynamical systems to very high precision, as required in scientific contexts. We propose using the novel multistage neural network approach with a spectrum-informed initialization to learn the residue from the previous stage, utilizing the spectral biases associated with neural networks to capture high frequency features in the residue, and successfully tackle the spectral bias of neural networks. This approach allows the neural network to fit target functions to double floating-point machine precision $O(10^{-16})$.