Sobolev Approximation of Deep ReLU Networks in Log-Barron Space

TL;DR

This paper introduces a new log-weighted Barron space to better understand how deep ReLU networks efficiently approximate functions in high-dimensional settings, highlighting the role of depth in reducing regularity constraints.

Contribution

The paper proposes a novel log-weighted Barron space, analyzes its properties, and establishes approximation bounds for deep ReLU networks, extending classical Barron space theory.

Findings

Functions in the new space can be approximated with explicit depth dependence.

Depth reduces regularity requirements for efficient approximation.

Results clarify why deep architectures perform well in high-dimensional problems.

Abstract

Universal approximation theorems show that neural networks can approximate any continuous function; however, the number of parameters may grow exponentially with the ambient dimension, so these results do not fully explain the practical success of deep models on high-dimensional data. Barron space theory addresses this: if a target function belongs to a Barron space, a two-layer network with $n$ parameters achieves an $O(n^{-1/2})$ approximation error in $L^2$. Yet classical Barron spaces $\mathscr{B}^{s+1}$ still require stronger regularity than Sobolev spaces $H^s$, and existing depth-sensitive results often assume constraints such as $sL \le 1/2$. In this paper, we introduce a log-weighted Barron space $\mathscr{B}^{\log}$, which requires a strictly weaker assumption than $\mathscr{B}^s$ for any $s>0$. For this new function space, we first study embedding properties and carry out a statistical analysis via the Rademacher complexity. Then we prove that functions in $\mathscr{B}^{\log}$ can be approximated by deep ReLU networks with explicit depth dependence. We then define a family $\mathscr{B}^{s,\log}$, establish approximation bounds in the $H^1$ norm, and identify maximal depth scales under which these rates are preserved. Our results clarify how depth reduces regularity requirements for efficient representation, offering a more precise explanation for the performance of deep architectures beyond the classical Barron setting, and for their stable use in high-dimensional problems used today.