Hypothesis testing near singularities and boundaries

TL;DR

This paper introduces a new distribution for hypothesis testing near model singularities and boundaries, where traditional chi-square approximations fail, improving test accuracy in complex models like evolutionary trees.

Contribution

The authors develop a novel distribution that accurately approximates the likelihood ratio statistic near singularities and boundaries, outperforming chi-square in such scenarios.

Findings

New distribution matches the likelihood ratio statistic asymptotically.

Outperforms chi-square in simulations for evolutionary tree models.

Improves hypothesis testing accuracy near model singularities.

Abstract

The likelihood ratio statistic, with its asymptotic $\chi^2$ distribution at regular model points, is often used for hypothesis testing. At model singularities and boundaries, however, the asymptotic distribution may not be $\chi^2$, as highlighted by recent work of Drton. Indeed, poor behavior of a $\chi^2$ for testing near singularities and boundaries is apparent in simulations, and can lead to conservative or anti-conservative tests. Here we develop a new distribution designed for use in hypothesis testing near singularities and boundaries, which asymptotically agrees with that of the likelihood ratio statistic. For two example trinomial models, arising in the context of inference of evolutionary trees, we show the new distributions outperform a $\chi^2$.