p-Values for Model Evaluation

TL;DR

This paper discusses the use and interpretation of p-values in model evaluation, clarifying their practical importance and providing insights into their calculation and application in data analysis.

Contribution

It offers a Bayesian perspective on p-values, explains various discrepancy variables, and evaluates their effectiveness in goodness-of-fit testing.

Findings

P-values are practically useful despite interpretational confusion.

Discrepancy variables can be used to compute p-values for model assessment.

Examples demonstrate the application of p-values in typical data analysis scenarios.

Abstract

Deciding whether a model provides a good description of data is often based on a goodness-of-fit criterion summarized by a p-value. Although there is considerable confusion concerning the meaning of p-values, leading to their misuse, they are nevertheless of practical importance in common data analysis tasks. We motivate their application using a Bayesian argumentation. We then describe commonly and less commonly known discrepancy variables and how they are used to define p-values. The distribution of these are then extracted for examples modeled on typical data analysis tasks, and comments on their usefulness for determining goodness-of-fit are given.