Error estimate for a universal function approximator of ReLU network with a local connection

TL;DR

This paper provides an error estimate for a local connection neural network architecture, which is more applicable than fully connected networks and can explain diverse neural network types like CNNs.

Contribution

It introduces an approximation error analysis for local connection neural networks, highlighting how depth and width influence performance.

Findings

Error estimate depends on depth and width parameters

Local connection networks can effectively explain CNNs

Provides theoretical foundation for neural network approximation error

Abstract

Neural networks have shown high successful performance in a wide range of tasks, but further studies are needed to improve its performance. We analyze the approximation error of the specific neural network architecture with a local connection and higher application than one with the full connection because the local-connected network can be used to explain diverse neural networks such as CNNs. Our error estimate depends on two parameters: one controlling the depth of the hidden layer, and the other, the width of the hidden layers.