Posterior Contraction for Sparse Neural Networks in Besov Spaces with Intrinsic Dimensionality

TL;DR

This paper proves that sparse Bayesian neural networks can adaptively achieve optimal posterior contraction rates over complex Besov spaces, effectively handling high-dimensional structured functions by leveraging intrinsic dimensionality.

Contribution

It establishes the theoretical optimality and adaptability of Bayesian neural networks with sparse priors for functions in Besov spaces, addressing high-dimensional challenges.

Findings

Achieves optimal posterior contraction rates for Besov space functions.

Demonstrates rate adaptation even with unknown smoothness levels.

Supports a broad class of structured functions, including additive and multiplicative Besov functions.

Abstract

This work establishes that sparse Bayesian neural networks achieve optimal posterior contraction rates over anisotropic Besov spaces and their hierarchical compositions. These structures reflect the intrinsic dimensionality of the underlying function, thereby mitigating the curse of dimensionality. Our analysis shows that Bayesian neural networks equipped with either sparse or continuous shrinkage priors attain the optimal rates which are dependent on the intrinsic dimension of the true structures. Moreover, we show that these priors enable rate adaptation, allowing the posterior to contract at the optimal rate even when the smoothness level of the true function is unknown. The proposed framework accommodates a broad class of functions, including additive and multiplicative Besov functions as special cases. These results advance the theoretical foundations of Bayesian neural networks and provide rigorous justification for their practical effectiveness in high-dimensional, structured estimation problems.