Tests of Linear Hypotheses using Indirect Information

TL;DR

This paper introduces new test statistics for multigroup linear hypotheses that leverage information sharing across groups, improving power in small sample settings while maintaining control over type I error.

Contribution

It develops a novel class of tests that adaptively incorporate information from other groups, enhancing power compared to standard $F$-tests in small-sample, high-dimensional contexts.

Findings

Proposed tests outperform $F$-tests in power in simulations.

The tests maintain exact type I error rates.

Empirical analysis shows improved detection of effects in educational data.

Abstract

In multigroup data settings with small within-group sample sizes, standard $F$-tests of group-specific linear hypotheses can have low power, particularly if the within-group sample sizes are not large relative to the number of explanatory variables. To remedy this situation, in this article we derive alternative test statistics based on information-sharing across groups. Each group-specific test has potentially much larger power than the standard $F$-test, while still exactly maintaining a target type I error rate if the hypothesis for the group is true. The proposed test for a given group uses a statistic that has optimal marginal power under a prior distribution derived from the data of the other groups. This statistic approaches the usual $F$-statistic as the prior distribution becomes more diffuse, but approaches a limiting "cone" test statistic as the prior distribution becomes extremely concentrated. We compare the power and $p$-values of the cone test to that of the $F$-test in some high-dimensional asymptotic scenarios. An analysis of educational outcome data is provided, demonstrating empirically that the proposed test is more powerful than the $F$-test.