Exact Minimum-Volume Confidence Set Intersection for Multinomial Outcomes

TL;DR

This paper develops a certified algorithm to determine whether minimum-volume confidence sets for multinomial outcomes intersect, improving the reliability of A/B testing and reinforcement learning analyses.

Contribution

It introduces a novel, geometric, and adaptive algorithm for certifying intersection of MVCs, extending to higher dimensions, with practical applications in data science.

Findings

Efficient algorithm for three-category MVC intersection testing.

Provably sound method with adaptive geometric partitioning.

Extension of the approach to higher-dimensional multinomial outcomes.

Abstract

Computation of confidence sets is central to data science and machine learning, serving as the workhorse of A/B testing and underpinning the operation and analysis of reinforcement learning algorithms. Among all valid confidence sets for the multinomial parameter, minimum-volume confidence sets (MVCs) are optimal in that they minimize average volume, but they are defined as level sets of an exact p-value that is discontinuous and difficult to compute. Rather than attempting to characterize the geometry of MVCs directly, this paper studies a practically motivated decision problem: given two observed multinomial outcomes, can one certify whether their MVCs intersect? We present a certified, tolerance-aware algorithm for this intersection problem. The method exploits the fact that likelihood ordering induces halfspace constraints in log-odds coordinates, enabling adaptive geometric partitioning of parameter space and computable lower and upper bounds on p-values over each cell. For three categories, this yields an efficient and provably sound algorithm that either certifies intersection, certifies disjointness, or returns an indeterminate result when the decision lies within a prescribed margin. We further show how the approach extends to higher dimensions. The results demonstrate that, despite their irregular geometry, MVCs admit reliable certified decision procedures for core tasks in A/B testing.