Hyena Neural Operator for Partial Differential Equations

TL;DR

The paper introduces Hyena, a neural operator with a long convolutional filter for efficiently solving partial differential equations, demonstrating improved accuracy and computational efficiency over existing methods.

Contribution

It presents a novel neural operator architecture called Hyena that uses a sub-quadratic, state space model-based long convolution for PDE solutions.

Findings

Hyena outperforms existing neural operators in accuracy.

Hyena demonstrates computational efficiency with sub-quadratic complexity.

Effective on Diffusion-Reaction and Navier-Stokes equations.

Abstract

Numerically solving partial differential equations typically requires fine discretization to resolve necessary spatiotemporal scales, which can be computationally expensive. Recent advances in deep learning have provided a new approach to solving partial differential equations that involves the use of neural operators. Neural operators are neural network architectures that learn mappings between function spaces and have the capability to solve partial differential equations based on data. This study utilizes a novel neural operator called Hyena, which employs a long convolutional filter that is parameterized by a multilayer perceptron. The Hyena operator is an operation that enjoys sub-quadratic complexity and state space model to parameterize long convolution that enjoys a global receptive field. This mechanism enhances the model's comprehension of the input's context and enables data-dependent weight for different partial differential equations instances. To measure how effective the layers are in solving partial differential equations, we conduct experiments on Diffusion-Reaction equation and Navier Stokes equation. Our findings indicate Hyena Neural operator can serve as an efficient and accurate model for learning partial differential equations solution operator. The data and code used can be found at: https://github.com/Saupatil07/Hyena-Neural-Operator