Te Test: A New Non-asymptotic T-test for Behrens-Fisher Problems

TL;DR

This paper introduces Te Test, a novel exact non-asymptotic t-test for the Behrens-Fisher problem, providing improved accuracy and confidence intervals compared to traditional methods, especially with unequal variances or sample sizes.

Contribution

The paper presents the first exact non-asymptotic t-test for the Behrens-Fisher problem, with theoretical analysis and superior performance demonstrated through simulations.

Findings

Te test achieves maximum degrees of freedom.

Te test provides shortest expected confidence interval length.

Simulation results show Te test outperforms conventional methods.

Abstract

The Behrens-Fisher Problem is a classical statistical problem. It is to test the equality of the means of two normal populations using two independent samples, when the equality of the population variances is unknown. Linnik (1968) has shown that this problem has no exact fixed-level tests based on the complete sufficient statistics. However, exact conventional solutions based on other statistics and approximate solutions based the complete sufficient statistics do exist. Existing methods are mainly asymptotic tests, and usually don't perform well when the variances or sample sizes differ a lot. In this paper, we propose a new method to find an exact t-test (Te) to solve this classical Behrens-Fisher Problem. Confidence intervals for the difference between two means are provided. We also use detailed analysis to show that Te test reaches the maximum of degree of freedom and to give a weak version of proof that Te test has the shortest confidence interval length expectation. Some simulations are performed to show the advantages of our new proposed method compared to available conventional methods like Welch's test, paired t-test and so on. We will also compare it to unconventional method, like two-stage test.