Neural Flow Operators can Approximate any Operator: Abstract Frameworks and Universal Approcimations

TL;DR

This paper introduces an abstract neural flow framework capable of universal approximation for both finite and infinite-dimensional spaces, unifying residual and plain neural network architectures.

Contribution

It provides the first universal approximation results for flow-based models between infinite-dimensional spaces and unifies residual and plain architectures within a flow-based framework.

Findings

Proves well-posedness and universal approximation properties for neural flows.

Establishes universal approximation for convolutional neural flow models.

Unifies residual and plain architectures via flow discretizations.

Abstract

We introduce an abstract neural flow framework for neural networks and neural operators. The framework contains two continuous-depth models, namely neural flows with composition and separation structures, and covers both finite-dimensional function approximation and infinite-dimensional operator approximation. We prove well-posedness and universal approximation properties for the corresponding neural flows, including, to the best of our knowledge, the first universal approximation result for flow-based models between infinite-dimensional spaces. We also obtain universal approximation results for convolutional neural flow models. Through suitable time discretizations, the composition structure recovers ResNet-type architectures, while the separation structure, via a splitting-based discretization, yields plain architectures. This gives a unified flow-based route to both residual and plain architectures for neural networks and neural operators with fully connected or convolutional linear layers.