Stratified Bootstrap Test Package
Ehsan Mohammadi, Fanghua Chen, Yizhou Cai, Yun Yang, Ting Fung Ma, Lu Zhou

TL;DR
The paper introduces the Stratified Bootstrap Test package, a nonparametric framework for assessing the stability and statistical significance of group-specific rankings in multivariate data, applicable to survey analysis and fairness auditing.
Contribution
It develops a novel stratified bootstrap methodology for both descriptive ranking stability assessment and formal hypothesis testing of group orderings, without relying on distributional assumptions.
Findings
Provides a nonparametric measure of ranking robustness
Enables formal testing of group ordering hypotheses
Applicable to survey, AI fairness, and response pattern analysis
Abstract
The Stratified Bootstrap Test (SBT) provides a nonparametric, resampling-based framework for assessing the stability of group-specific ranking patterns in multivariate survey or rating data. By repeatedly resampling observations and examining whether a group's top-ranked items remain among the highest-scoring categories across bootstrap samples, SBT quantifies ranking robustness through a non-containment index. In parallel, the stratified bootstrap test extends this framework to formal statistical inference by testing ordering hypotheses among population means. Through resampling within groups, the method approximates the null distribution of ranking-based test statistics without relying on distributional assumptions. Together, these techniques enable both descriptive and inferential evaluation of ranking consistency, detection of aberrant or adversarial response patterns, and rigorous…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPsychometric Methodologies and Testing · Explainable Artificial Intelligence (XAI) · Ethics and Social Impacts of AI
