Improved small-sample inference for functions of parameters in the k-sample multinomial problem

TL;DR

This paper introduces an exact inference method for functions of parameters in the k-sample multinomial problem, addressing challenges in small samples and nondifferentiable functions, with a Monte Carlo implementation ensuring practical applicability.

Contribution

It develops a general, exact inference approach with a Monte Carlo method for functions of multinomial probabilities, improving small-sample inference accuracy.

Findings

Exact p-value bounds type I error rate.

Confidence intervals achieve nominal coverage.

Monte Carlo method is consistent with increasing iterations.

Abstract

When the target parameter for inference is a real-valued, continuous function of probabilities in the $k$-sample multinomial problem, variance estimation may be challenging. In small samples or when the function is nondifferentiable at the true parameter, methods like the nonparametric bootstrap or delta method may perform poorly. We develop an exact inference method that applies to this general situation. We prove that our proposed exact p-value correctly bounds the type I error rate and the associated confidence intervals provide at least nominal coverage; however, they are generally difficult to implement. Thus, we propose a Monte Carlo implementation to estimate the exact p-value and confidence intervals that we show to be consistent as the number of iterations grows. Our approach is general in that it applies to any real-valued continuous function of multinomial probabilities from an arbitrary number of samples and with different numbers of categories.