Universal Inference

TL;DR

This paper introduces a universal method for hypothesis testing and confidence set construction that guarantees finite sample validity across a wide range of models, including irregular and nonparametric ones, using a simple split likelihood ratio test.

Contribution

The paper presents a novel, simple split likelihood ratio test that provides finite sample guarantees for hypothesis testing and confidence sets in both regular and irregular models, including nonparametric cases.

Findings

Works for any parametric model and some nonparametric models

Provides finite sample guarantees without regularity conditions

Enables sequential testing with anytime-valid p-values

Abstract

We propose a general method for constructing hypothesis tests and confidence sets that have finite sample guarantees without regularity conditions. We refer to such procedures as "universal." The method is very simple and is based on a modified version of the usual likelihood ratio statistic, that we call "the split likelihood ratio test" (split LRT). The method is especially appealing for irregular statistical models. Canonical examples include mixture models and models that arise in shape-constrained inference. Constructing tests and confidence sets for such models is notoriously difficult. Typical inference methods, like the likelihood ratio test, are not useful in these cases because they have intractable limiting distributions. In contrast, the method we suggest works for any parametric model and also for some nonparametric models. The split LRT can also be used with profile likelihoods to deal with nuisance parameters, and it can also be run sequentially to yield anytime-valid $p$-values and confidence sequences.