Triangulating PL functions and the existence of efficient ReLU DNNs

TL;DR

This paper demonstrates that any piecewise linear function with compact support can be represented as a sum of simplex functions derived from triangulations, leading to efficient ReLU neural networks for such functions.

Contribution

It introduces a novel triangulation-based method to represent piecewise linear functions, providing a new proof for the existence of efficient universal ReLU neural networks.

Findings

Representation of piecewise linear functions via simplex functions

Elementary proof of universal ReLU networks for bounded complexity functions

Efficient neural network constructions for piecewise linear functions

Abstract

We show that every piecewise linear function $f:R^d \to R$ with compact support a polyhedron $P$ has a representation as a sum of so-called `simplex functions'. Such representations arise from degree 1 triangulations of the relative homology class (in $R^{d+1}$) bounded by $P$ and the graph of $f$, and give a short elementary proof of the existence of efficient universal ReLU neural networks that simultaneously compute all such functions $f$ of bounded complexity.