Scale-Insensitive Neural Network Significance Tests

TL;DR

This paper introduces a scale-insensitive framework for neural network significance testing that relaxes previous assumptions, allowing for broader applicability and maintaining optimal convergence rates.

Contribution

It generalizes existing methods by replacing metric entropy with Rademacher complexity, weakening smoothness assumptions, and using moment bounds for sieve space construction.

Findings

Achieves optimal convergence rates

Establishes valid asymptotic distributions

Handles unbounded weights and general activation functions

Abstract

This paper develops a scale-insensitive framework for neural network significance testing, substantially generalizing existing approaches through three key innovations. First, we replace metric entropy calculations with Rademacher complexity bounds, enabling the analysis of neural networks without requiring bounded weights or specific architectural constraints. Second, we weaken the regularity conditions on the target function to require only Sobolev space membership $H^s([-1,1]^d)$ with $s > d/2$, significantly relaxing previous smoothness assumptions while maintaining optimal approximation rates. Third, we introduce a modified sieve space construction based on moment bounds rather than weight constraints, providing a more natural theoretical framework for modern deep learning practices. Our approach achieves these generalizations while preserving optimal convergence rates and establishing valid asymptotic distributions for test statistics. The technical foundation combines localization theory, sharp concentration inequalities, and scale-insensitive complexity measures to handle unbounded weights and general Lipschitz activation functions. This framework better aligns theoretical guarantees with contemporary deep learning practice while maintaining mathematical rigor.