Polynomial-Spline Neural Networks with Exact Integrals

TL;DR

This paper introduces a neural network architecture combining polynomial mixture-of-experts with spline basis functions, enabling exact integration and improved consistency in solving variational problems.

Contribution

The paper presents a novel neural network architecture that achieves exact integral computation and consistency for variational problems, combining polynomial mixture models with free knot B-splines.

Findings

Exact integration of moments and derivatives achieved

Convergence rates match approximation theory expectations

Successful application to regression and variational problems

Abstract

Using neural networks to solve variational problems, and other scientific machine learning tasks, has been limited by a lack of consistency and an inability to exactly integrate expressions involving neural network architectures. We address these limitations by formulating a novel neural network architecture that combines a polynomial mixture-of-experts model with free knot B1-spline basis functions. Effectively, our architecture performs piecewise polynomial approximation on each cell of a trainable partition of unity. Our architecture exhibits both $h$- and $p$- refinement for regression problems at the convergence rates expected from approximation theory, allowing for consistency in solving variational problems. Moreover, this architecture, its moments, and its partial derivatives can all be integrated exactly, obviating a reliance on sampling or quadrature and enabling error-free computation of variational forms. We demonstrate the success of our network on a range of regression and variational problems that illustrate the consistency and exact integrability of our network architecture.