# A New Approach to the Nonparametric Behrens–Fisher Problem With Compatible Confidence Intervals

## TL;DR

This paper introduces a new statistical method for comparing two groups when their distributions differ, offering better accuracy and confidence intervals.

## Contribution

A novel nonparametric test for the Behrens–Fisher problem with improved type-I error control and compatible confidence intervals.

## Key findings

- The proposed test maintains accurate type-I error rates even with small or unbalanced samples.
- It outperforms the Brunner–Munzel test in controlling type-I errors at low significance levels.
- Compatible confidence intervals show improved coverage compared to existing methods.

## Abstract

We propose a new method to address the nonparametric Behrens–Fisher problem, allowing for unequal distribution functions across the two samples. The procedure tests the null hypothesis H0:θ=1/2, where θ=P(X<Y)+1/2P(X=Y) denotes the Mann–Whitney effect. Apart from the trivial case of one‐point distributions, no restrictions are imposed on the underlying data distribution. The test is derived by evaluating the ratio of the true variance σN2 of the Mann–Whitney effect estimator θ^N to its theoretical maximum, as derived from the Birnbaum–Klose inequality. Through simulations, we demonstrate that the proposed test effectively controls the type‐I error rate under various conditions, including small and unbalanced sample sizes, and different data‐generating mechanisms. Notably, it provides better control of the type‐I error rate than the widely used Brunner–Munzel test, particularly at small significance levels such as α=0.005. We further construct range‐preserving compatible confidence intervals and show that they exhibit improved coverage compared to the confidence intervals compatible to the Brunner–Munzel test. Finally, we illustrate the application of the method in a clinical trial example.