The Adequate Bootstrap

TL;DR

This paper proposes a new method for assessing model adequacy by focusing on the usefulness of the model despite its known imperfections, using bootstrap techniques to evaluate parameter uncertainty at a suitable sample size.

Contribution

It introduces a bootstrap-based approach to evaluate model adequacy from a practical perspective, emphasizing usefulness over strict correctness.

Findings

The method effectively measures parameter uncertainty caused by model limitations.

It allows inference based on a sample size where the model is not rejected.

The approach provides a more relevant assessment of model usefulness.

Abstract

There is a fundamental disconnect between what is tested in a model adequacy test, and what we would like to test. The usual approach is to test the null hypothesis "Model M is the true model." However, Model M is never the true model. A model might still be useful even if we have enough data to reject it. In this paper, we present a technique to assess the adequacy of a model from the philosophical standpoint that we know the model is not true, but we want to know if it is useful. Our solution to this problem is to measure the parameter uncertainty in our estimates caused by the model uncertainty. We use bootstrap inference on samples of a smaller size, for which the model cannot be rejected. We use a model adequacy test to choose a bootstrap size with limited probability of rejecting the model and perform inference for samples of this size based on a nonparametric bootstrap. Our idea is that if we base our inference on a sample size at which we do not reject the model, then we should be happy with this inference, because we would have been confident in it if our original dataset had been this size.