Dimension-independent learning rates for high-dimensional classification problems

TL;DR

This paper demonstrates that neural networks can approximate high-dimensional classification functions with bounded weights, providing dimension-independent learning rates and analyzing the impact of regularity conditions on decision boundaries.

Contribution

It extends existing results to show neural networks can approximate $RBV^2$ functions with bounded weights and quantifies estimation rates in high dimensions.

Findings

Neural networks can approximate $RBV^2$ functions without the curse of dimensionality.

Bounded weights neural networks can effectively approximate classification functions.

Regularity conditions influence the decision boundary complexity and approximation quality.

Abstract

We study the problem of approximating and estimating classification functions that have their decision boundary in the $RBV^2$ space. Functions of $RBV^2$ type arise naturally as solutions of regularized neural network learning problems and neural networks can approximate these functions without the curse of dimensionality. We modify existing results to show that every $RBV^2$ function can be approximated by a neural network with bounded weights. Thereafter, we prove the existence of a neural network with bounded weights approximating a classification function. And we leverage these bounds to quantify the estimation rates. Finally, we present a numerical study that analyzes the effect of different regularity conditions on the decision boundaries.