Computing the confidence levels for a root-mean-square test of goodness-of-fit

TL;DR

This paper introduces efficient algorithms for computing confidence levels of a modified chi-squared goodness-of-fit test that avoids problematic divisions, enhancing the test's practicality with computer assistance.

Contribution

It presents novel black-box algorithms for asymptotic confidence level calculation and discusses Monte Carlo methods for exact levels, improving the classical chi-squared test.

Findings

Algorithms enable efficient confidence level computation

Monte Carlo simulation can provide exact confidence levels

The method simplifies goodness-of-fit testing without rebinning

Abstract

The classic chi-squared statistic for testing goodness-of-fit has long been a cornerstone of modern statistical practice. The statistic consists of a sum in which each summand involves division by the probability associated with the corresponding bin in the distribution being tested for goodness-of-fit. Typically this division should precipitate rebinning to uniformize the probabilities associated with the bins, in order to make the test reasonably powerful. With the now widespread availability of computers, there is no longer any need for this. The present paper provides efficient black-box algorithms for calculating the asymptotic confidence levels of a variant on the classic chi-squared test which omits the problematic division. In many circumstances, it is also feasible to compute the exact confidence levels via Monte Carlo simulation.