Statistical Validation of Computer Models: Global and Subdomain Hypothesis Testing

TL;DR

This paper introduces a new statistical framework and test, FMMT, for validating computer models against real data, emphasizing both global and localized discrepancies.

Contribution

It develops the Fourier Maximum Modulus Test (FMMT) using kernel ridge regression and frequency-domain analysis for rigorous model validation.

Findings

FMMT achieves high power in detecting discrepancies.

FMMT accurately controls Type I error.

FMMT is sensitive to localized model-data mismatches.

Abstract

Computer simulations play an important role in scientific discovery and engineering innovation. Reliable computer models enable virtual experimentation that reduces the need for costly and time-consuming physical testing. However, the credibility of such models hinges on rigorous statistical validation against real-world data. This paper develops a formal frequentist framework for both global and subdomain validation of computer models. We propose the Fourier Maximum Modulus Test (FMMT), which leverages kernel ridge regression (KRR) to estimate the discrepancy between the computer model and the physical process, followed by a frequency-domain test based on weighted generalized Fourier coefficients. The theoretical analysis establishes the asymptotic normality of these coefficients, allowing for closed-form p-values. Simulation studies and a shear-layer experiment demonstrate that FMMT achieves high power, accurate Type I error control, and strong sensitivity to localized discrepancies.