ReLU neural network approximation to piecewise constant functions

TL;DR

This paper demonstrates that three-layer ReLU neural networks can effectively approximate piecewise constant functions with unknown interfaces, providing error bounds and explicit formulas for convex discontinuity interfaces.

Contribution

It establishes the approximation capabilities of shallow ReLU networks for piecewise constant functions and offers explicit formulas when interfaces are convex.

Findings

Three-layer ReLU networks can approximate piecewise constant functions with controlled error.

Explicit formulas are derived for convex interface cases.

Error bounds depend on the approximation accuracy of the interface by hyperplanes.

Abstract

This paper studies the approximation property of ReLU neural networks (NNs) to piecewise constant functions with unknown interfaces in bounded regions in $\mathbb{R}^d$. Under the assumption that the discontinuity interface $\Gamma$ may be approximated by a connected series of hyperplanes with a prescribed accuracy $\varepsilon >0$, we show that a three-layer ReLU NN is sufficient to accurately approximate any piecewise constant function and establish its error bound. Moreover, if the discontinuity interface is convex, an analytical formula of the ReLU NN approximation with exact weights and biases is provided.