A More Robust t-Test

TL;DR

This paper introduces a new robust t-test for small samples and clustered data, combining extreme value theory and normal approximation to improve size control and accuracy over traditional methods.

Contribution

It develops a novel t-test that better handles heavy tails and heterogeneity in small samples, especially under clustering, outperforming existing tests.

Findings

The new test controls size more effectively in small samples.

It provides a refinement over the full sample t-test with more than two moments.

The bootstrapped t-test does not offer the same refinement.

Abstract

Standard inference about a scalar parameter estimated via GMM amounts to applying a t-test to a particular set of observations. If the number of observations is not very large, then moderately heavy tails can lead to poor behavior of the t-test. This is a particular problem under clustering, since the number of observations then corresponds to the number of clusters, and heterogeneity in cluster sizes induces a form of heavy tails. This paper combines extreme value theory for the smallest and largest observations with a normal approximation for the average of the remaining observations to construct a more robust alternative to the t-test. The new test is found to control size much more successfully in small samples compared to existing methods. Analytical results in the canonical inference for the mean problem demonstrate that the new test provides a refinement over the full sample t-test under more than two but less than three moments, while the bootstrapped t-test does not.