Universal Approximation of Continuous Functionals on Compact Subsets via Linear Measurements and Scalar Nonlinearities

TL;DR

This paper demonstrates that continuous functionals on compact sets can be universally approximated using a simple two-step process involving linear measurements followed by scalar nonlinearities, supporting common practices in operator learning.

Contribution

It establishes a universal approximation theorem for continuous functionals using linear measurements and nonlinearities, extending to Banach space-valued maps and providing theoretical backing for existing operator learning methods.

Findings

Any continuous functional can be approximated uniformly by models with linear measurements and nonlinearities.

Extension of approximation results to maps with Banach space values and finite-rank approximations.

Provides theoretical justification for the 'measure, apply nonlinearities, then combine' approach in operator learning.

Abstract

We study universal approximation of continuous functionals on compact subsets of products of Hilbert spaces. We prove that any such functional can be uniformly approximated by models that first take finitely many continuous linear measurements of the inputs and then combine these measurements through continuous scalar nonlinearities. We also extend the approximation principle to maps with values in a Banach space, yielding finite-rank approximations. These results provide a compact-set justification for the common ``measure, apply scalar nonlinearities, then combine'' design pattern used in operator learning and imaging.