Model Calibration and Validation From A Statistical Inference Perspective

TL;DR

This paper offers a rigorous, statistically grounded framework for model calibration and validation, addressing inconsistencies in current practices and aiming to improve their scientific rigor across transport and other fields.

Contribution

It provides a unified, formal formulation of calibration and validation rooted in statistical inference, clarifying concepts and suggesting improvements for current methodologies.

Findings

Identifies inconsistencies in existing calibration and validation practices.

Proposes a unified, statistically sound framework for model validation.

Highlights the importance of rigorous statistical methods in model assessment.

Abstract

Despite the general consensus in transport research community that model calibration and validation are necessary to enhance model predictive performance, there exist significant inconsistencies in the literature. This is primarily due to a lack of consistent definitions, and a unified and statistically sound framework. In this paper, we provide a general and rigorous formulation of the model calibration and validation problem, and highlight its relation to statistical inference. We also conduct a comprehensive review of the steps and challenges involved, as well as point out inconsistencies, before providing suggestions on improving the current practices. This paper is intended to help the practitioners better understand the nature of model calibration and validation, and to promote statistically rigorous and correct practices. Although the examples are drawn from a transport research background - and that is our target audience - the content in this paper is equally applicable to other modelling contexts.