Semialgebraic Neural Networks: From roots to representations

TL;DR

Semialgebraic Neural Networks (SANNs) are a novel architecture capable of representing any bounded semialgebraic function, including discontinuous ones, by encoding their graphs and executing a homotopy continuation method, with applications demonstrated.

Contribution

Introduces SANNs, a neural network architecture that can exactly represent semialgebraic functions, including discontinuous ones, using a homotopy continuation approach.

Findings

SANNs can represent any bounded semialgebraic function.

SANNs can handle discontinuous functions by executing continuation on each component.

Networks can be trained with traditional deep-learning techniques.

Abstract

Many numerical algorithms in scientific computing -- particularly in areas like numerical linear algebra, PDE simulation, and inverse problems -- produce outputs that can be represented by semialgebraic functions; that is, the graph of the computed function can be described by finitely many polynomial equalities and inequalities. In this work, we introduce Semialgebraic Neural Networks (SANNs), a neural network architecture capable of representing any bounded semialgebraic function, and computing such functions up to the accuracy of a numerical ODE solver chosen by the programmer. Conceptually, we encode the graph of the learned function as the kernel of a piecewise polynomial selected from a class of functions whose roots can be evaluated using a particular homotopy continuation method. We show by construction that the SANN architecture is able to execute this continuation method, thus evaluating the learned semialgebraic function. Furthermore, the architecture can exactly represent even discontinuous semialgebraic functions by executing a continuation method on each connected component of the target function. Lastly, we provide example applications of these networks and show they can be trained with traditional deep-learning techniques.