
TL;DR
This paper introduces a general method for conducting permutation tests on complex survey data by adjusting for the sample design, ensuring valid p-value estimation in hypothesis testing.
Contribution
It proposes a pseudo-permutation test that accounts for complex sample design features, extending permutation testing to survey data with clustered residuals.
Findings
The method improves p-value accuracy over traditional permutation tests ignoring design.
Simulations show the necessity of accounting for sample design features.
The approach is applicable to various complex survey structures.
Abstract
Permutation tests are a distribution free way of performing hypothesis tests. These tests rely on the condition that the observed data are exchangeable among the groups being tested under the null hypothesis. This assumption is easily satisfied for data obtained from a simple random sample or a controlled study after simple adjustments to the data, but there is no general method for adjusting survey data collected using a complex sample design to allow for permutation tests. In this article, we propose a general method for performing a pseudo- permutation test that accounts for the complex sample design. The proposed method is not a true permutation test in that the new values do not come from the set of observed values in general, but of an expanded set of values satisfying a random-effects model on the clustered residuals. Tests using a simulated population comparing the performance…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
