Residual permutation test for regression coefficient testing
Kaiyue Wen, Tengyao Wang, Yuhao Wang

TL;DR
This paper introduces the residual permutation test (RPT), a new method for testing regression coefficients in high-dimensional linear models that maintains finite-sample validity and demonstrates strong power under heavy-tailed noise.
Contribution
The paper proposes RPT, a novel permutation-based testing procedure that achieves finite-sample size control and near-optimal power in high-dimensional linear regression.
Findings
RPT maintains size control under fixed design with exchangeable noise.
RPT is asymptotically powerful for heavy-tailed noise with bounded moments.
Numerical results show RPT performs well across various noise distributions.
Abstract
We consider the problem of testing whether a single coefficient is equal to zero in linear models when the dimension of covariates can be up to a constant fraction of sample size . In this regime, an important topic is to propose tests with finite-sample valid size control without requiring the noise to follow strong distributional assumptions. In this paper, we propose a new method, called residual permutation test (RPT), which is constructed by projecting the regression residuals onto the space orthogonal to the union of the column spaces of the original and permuted design matrices. RPT can be proved to achieve finite-population size validity under fixed design with just exchangeable noises, whenever . Moreover, RPT is shown to be asymptotically powerful for heavy tailed noises with bounded -th order moment when the true coefficient is at least of order…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsStatistical Distribution Estimation and Applications · Bayesian Methods and Mixture Models · Random Matrices and Applications
