Optimal Sample Size Planning for the Wilcoxon-Mann-Whitney-Test

TL;DR

This paper introduces a unified method for sample size planning for the Wilcoxon-Mann-Whitney test across various data types, providing formulas for optimal sample sizes and allocation ratios based on distribution characteristics.

Contribution

It presents a comprehensive approach that covers multiple data types and derives formulas for optimal sample size and allocation, including conditions for balanced designs.

Findings

Balanced design is optimal for certain distributions.

Optimal allocation depends on the ratio of variances under the alternative.

The method applies to metric, categorical, and dichotomous data.

Abstract

There are many different proposed procedures for sample size planning for the Wilcoxon-Mann-Whitney test at given type-I and type-II error rates $\alpha$ and $\beta$, respectively. Most methods assume very specific models or types of data in order to simplify calculations (for example, ordered categorical or metric data, location shift alternatives, etc.). We present a unified approach that covers metric data with and without ties, count data, ordered categorical data, and even dichotomous data. For that, we calculate the unknown theoretical quantities such as the variances under the null and relevant alternative hypothesis by considering the following `synthetic data' approach. We evaluate data whose empirical distribution functions match with the theoretical distribution functions involved in the computations of the unknown theoretical quantities. Then well-known relations for the ranks of the data are used for the calculations. In addition to computing the necessary sample size $N$ for a fixed allocation proportion $t = n_1/N$, where $n_1$ is the sample size in the first group and $N = n_1 + n_2$ is the total sample size, we provide an interval for the optimal allocation rate $t$ which minimizes the total sample size $N$. It turns out that for certain distributions, a balanced design is optimal. We give a characterization of these distributions. Furthermore we show that the optimal choice of $t$ depends on the ratio of the two variances which determine the variance of the Wilcoxon-Mann-Whitney statistic under the alternative. This is different from an optimal sample size allocation in case of the normal distribution model.