Shallow neural network representation of polynomials

TL;DR

This paper demonstrates that shallow neural networks can efficiently represent multivariate polynomials and achieve near-optimal convergence rates for univariate regression functions, highlighting their expressive power and approximation capabilities.

Contribution

It provides explicit constructions for representing multivariate polynomials with shallow networks and establishes near-minimax optimal convergence rates for univariate function approximation.

Findings

Shallow networks can represent multivariate polynomials of degree R with width 2(R+d)^d.

Achieves near minimax optimal convergence rates for univariate regression using shallow networks.

Provides theoretical bounds on the expressive power of shallow neural networks.

Abstract

We show that $d$-variate polynomials of degree $R$ can be represented on $[0,1]^d$ as shallow neural networks of width $2(R+d)^d$. Also, by SNN representation of localized Taylor polynomials of univariate $C^\beta$-smooth functions, we derive for shallow networks the minimax optimal rate of convergence, up to a logarithmic factor, to unknown univariate regression function.