On approximating the distributions of goodness-of-fit test statistics based on the empirical distribution function: The case of unknown parameters

TL;DR

This paper examines the challenges in accurately approximating the distributions of goodness-of-fit test statistics when parameters are unknown, highlighting the risks of neglecting re-estimation in Monte Carlo simulations which can lead to overly conservative tests.

Contribution

It demonstrates the importance of re-estimating unknown parameters in Monte Carlo simulations for goodness-of-fit tests and shows that neglecting this can cause significant inaccuracies.

Findings

Ignoring parameter re-estimation leads to overly conservative tests

The impact of neglecting re-estimation can be dramatic and persistent

Incorrect approximation may not improve with larger sample sizes

Abstract

This paper discusses some problems possibly arising when approximating via Monte-Carlo simulations the distributions of goodness-of-fit test statistics based on the empirical distribution function. We argue that failing to re-estimate unknown parameters on each simulated Monte-Carlo sample -- and thus avoiding to employ this information to build the test statistic -- may lead to wrong, overly-conservative testing. Furthermore, we present a simple example suggesting that the impact of this possible mistake may turn out to be dramatic and does not vanish as the sample size increases.