Partition of unity networks: deep hp-approximation

TL;DR

Partition of unity networks (POUnets) integrate partition of unity and polynomial approximation directly into neural network architecture, achieving efficient high-dimensional approximation and outperforming traditional MLPs especially for functions with discontinuities.

Contribution

This paper introduces POUnets, a novel neural network architecture that incorporates hp-approximation principles, breaking the curse of dimensionality and enabling efficient approximation of complex functions.

Findings

POUnets achieve hp-convergence for smooth functions.

POUnets outperform MLPs on functions with many discontinuities.

The architecture size does not scale exponentially with dimension.

Abstract

Approximation theorists have established best-in-class optimal approximation rates of deep neural networks by utilizing their ability to simultaneously emulate partitions of unity and monomials. Motivated by this, we propose partition of unity networks (POUnets) which incorporate these elements directly into the architecture. Classification architectures of the type used to learn probability measures are used to build a meshfree partition of space, while polynomial spaces with learnable coefficients are associated to each partition. The resulting hp-element-like approximation allows use of a fast least-squares optimizer, and the resulting architecture size need not scale exponentially with spatial dimension, breaking the curse of dimensionality. An abstract approximation result establishes desirable properties to guide network design. Numerical results for two choices of architecture demonstrate that POUnets yield hp-convergence for smooth functions and consistently outperform MLPs for piecewise polynomial functions with large numbers of discontinuities.