Enhancing Solutions for Complex PDEs: Introducing Complementary Convolution and Equivariant Attention in Fourier Neural Operators

TL;DR

This paper introduces a hierarchical Fourier neural operator with complementary convolution and attention mechanisms, significantly improving the solution of complex PDEs with rapid coefficient variations and oscillations.

Contribution

It proposes a novel hierarchical Fourier neural operator with convolution-residual layers and attention mechanisms to enhance PDE solving capabilities in complex scenarios.

Findings

Achieves superior performance on multiscale elliptic and Navier-Stokes equations

Addresses low-frequency approximation limitations of traditional FNO

Effective in scenarios with rapid coefficient changes

Abstract

Neural operators improve conventional neural networks by expanding their capabilities of functional mappings between different function spaces to solve partial differential equations (PDEs). One of the most notable methods is the Fourier Neural Operator (FNO), which draws inspiration from Green's function method and directly approximates operator kernels in the frequency domain. However, after empirical observation followed by theoretical validation, we demonstrate that the FNO approximates kernels primarily in a relatively low-frequency domain. This suggests a limited capability in solving complex PDEs, particularly those characterized by rapid coefficient changes and oscillations in the solution space. Such cases are crucial in specific scenarios, like atmospheric convection and ocean circulation. To address this challenge, inspired by the translation equivariant of the convolution kernel, we propose a novel hierarchical Fourier neural operator along with convolution-residual layers and attention mechanisms to make them complementary in the frequency domain to solve complex PDEs. We perform experiments on forward and reverse problems of multiscale elliptic equations, Navier-Stokes equations, and other physical scenarios, and find that the proposed method achieves superior performance in these PDE benchmarks, especially for equations characterized by rapid coefficient variations.