Integral representations of shallow neural network with Rectified Power Unit activation function

TL;DR

This paper derives integral formulas for shallow neural networks with Rectified Power Unit activations, characterizing their function representation capabilities in univariate and multivariate cases.

Contribution

It provides the first integral representation formulas for RePU-activated shallow networks, including univariate and multivariate function classes.

Findings

Univariate RePU shallow networks' representation capabilities are characterized.

Multidimensional RePU networks can represent functions with bounded norm and unbounded width.

The paper establishes integral formulas linking network parameters to function spaces.

Abstract

In this effort, we derive a formula for the integral representation of a shallow neural network with the Rectified Power Unit activation function. Mainly, our first result deals with the univariate case of representation capability of RePU shallow networks. The multidimensional result in this paper characterizes the set of functions that can be represented with bounded norm and possibly unbounded width.