Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums

TL;DR

This paper introduces Residual-Permuted Sums (RPS), a distribution-free method for constructing confidence regions in linear regression that relaxes symmetry assumptions and guarantees finite-sample coverage and consistency.

Contribution

It presents RPS as a novel permutation-based approach that extends SPS, providing finite-sample guarantees without requiring symmetric noise distributions.

Findings

RPS achieves exact finite-sample coverage probabilities.

RPS confidence regions are strongly consistent and shrink to true parameters.

Numerical experiments validate the effectiveness of RPS.

Abstract

This letter studies a distribution-free, finite-sample data perturbation (DP) method, the Residual-Permuted Sums (RPS), which is an alternative of the Sign-Perturbed Sums (SPS) algorithm, to construct confidence regions. While SPS assumes independent (but potentially time-varying) noise terms which are symmetric about zero, RPS gets rid of the symmetricity assumption, but assumes i.i.d. noises. The main idea is that RPS permutes the residuals instead of perturbing their signs. This letter introduces RPS in a flexible way, which allows various design-choices. RPS has exact finite sample coverage probabilities and we provide the first proof that these permutation-based confidence regions are uniformly strongly consistent under general assumptions. This means that the RPS regions almost surely shrink around the true parameters as the sample size increases. The ellipsoidal outer-approximation (EOA) of SPS is also extended to RPS, and the effectiveness of RPS is validated by numerical experiments, as well.