E-values for k-Sample Tests With Exponential Families

TL;DR

This paper introduces and compares e-variables for k-sample tests within exponential families, providing theoretical insights and efficient algorithms, with results showing their similar behavior under small effects and family-dependent growth rates.

Contribution

The paper develops new e-variables for k-sample testing in exponential families, extending previous methods and providing practical algorithms for their computation.

Findings

E-variables behave similarly under small effects across families.

For Gaussian and Poisson, certain e-variables coincide.

Family-dependent differences in growth rates of e-variables are observed.

Abstract

We develop and compare e-variables for testing whether $k$ samples of data are drawn from the same distribution, the alternative being that they come from different elements of an exponential family. We consider the GRO (growth-rate optimal) e-variables for (1) a `small' null inside the same exponential family, and (2) a `large' nonparametric null, as well as (3) an e-variable arrived at by conditioning on the sum of the sufficient statistics. (2) and (3) are efficiently computable, and extend ideas from Turner et al. [2021] and Wald [1947] respectively from Bernoulli to general exponential families. We provide theoretical and simulation-based comparisons of these e-variables in terms of their logarithmic growth rate, and find that for small effects all four e-variables behave surprisingly similarly; for the Gaussian location and Poisson families, e-variables (1) and (3) coincide; for Bernoulli, (1) and (2) coincide; but in general, whether (2) or (3) grows faster under the alternative is family-dependent. We furthermore discuss algorithms for numerically approximating (1).