Towards a unified theory for testing statistical hypothesis: Multinormal mean with nuisance covariance matrix

TL;DR

This paper develops a unified framework for testing hypotheses about the mean of a multinormal distribution with unknown covariance, reconciling Bayesian, frequentist, and Fisherian approaches, and analyzing the properties of likelihood ratio and union-intersection tests.

Contribution

It introduces a comprehensive framework that unifies different statistical testing philosophies for multinormal means with nuisance covariance, addressing issues with p-values and test admissibility.

Findings

Likelihood ratio and union-intersection tests are not proper Bayes tests.

The tests are power-dominated for certain alternatives.

Neither Fisher's nor Neyman-Pearson's criteria alone are sufficient.

Abstract

Under a multinormal distribution with an arbitrary unknown covariance matrix, the main purpose of this paper is to propose a framework to achieve the goal of reconciliation of Bayesian, frequentist, and Fisher's reporting $p$-values, Neyman-Pearson's optimal theory and Wald's decision theory for the problems of testing mean against restricted alternatives (closed convex cones). To proceed, the tests constructed via the likelihood ratio (LR) and the union-intersection (UI) principles are studied. For the problems of testing against restricted alternatives, first, we show that the LRT and the UIT are not the proper Bayes tests, however, they are shown to be the integrated LRT and the integrated UIT, respectively. For the problem of testing against the positive orthant space alternative, both the null distributions of the LRT and the UIT depend on the unknown nuisance covariance matrix. Hence we have difficulty adopting Fisher's approach to reporting $p$-values. On the other hand, according to the definition of the level of significance, both the LRT and the UIT are shown to be power-dominated by the corresponding LRT and UIT for testing against the half-space alternative, respectively. Hence, both the LRT and the UIT are $\alpha$-inadmissible, these results are against the common statistical sense. Neither Fisher's approach of reporting $p$-values alone nor Neyman-Pearson's optimal theory for power function alone is a satisfactory criterion for evaluating the performance of tests. Wald's decision theory via $d$-admissibility may shed light on resolving these challenging issues of imposing the balance between type 1 error and power.