Rethinking Model Transferability: Validity Domains as a New Approach to Delineate the Limits of Bloom Date Projections

TL;DR

This paper introduces validity domains to assess how well models can predict cherry blossom bloom dates under new climate conditions.

Contribution

The study introduces validity domains as a novel framework to evaluate model transferability by considering both extrapolation distance and environmental gradients.

Findings

Process-based models show broader validity when calibrated in colder environments.

Machine learning models have narrower but more consistent validity across temperature gradients.

Model reliability depends on both model structure and calibration environment.

Abstract

Accurately predicting future events under novel environmental conditions is a central challenge in modeling, especially when no validation data are available. While model transferability is often discussed through the concept of a “forecast horizon,” we expand this framework by introducing the concept of “validity domains.” These consider not only the extrapolation distance from the calibration data but also the absolute position of calibration and application conditions along an environmental gradient. Using phenological observations from Japanese Yoshino cherry (Prunus × yedoensis) across a climate gradient in Japan, we calibrated process‐based and machine learning models for each of 48 locations and validated them with data from all other locations. Interpolating model performance metrics yielded a continuous synthetic surface of predictive accuracy across the full observed temperature range, from which we delineated model‐specific validity domains and assessed how transferability depends on both model type and calibration environment. Our findings show that process‐based models retain broader validity when calibrated in colder environments but degrade in warmer settings. In contrast, machine learning models exhibit narrower but more consistent validity across the gradient. These systematic differences reveal that the location of calibration and the structure of the model fundamentally shape its reliability under new conditions. By identifying where prediction errors remain below a context‐specific validity threshold, our approach provides a robust framework for assessing model applicability under shifting climate conditions. Mapping validity domains offers practical guidance for model selection and allows quantifying how far models can be pushed before their predictions become unreliable.

Accurately predicting future phenology events in novel environmental conditions is a key challenge in modelling. Using first bloom observations of the Japanese Yoshino cherry across a temperature gradient, we calibrated process‐based and machine learning models at multiple locations and cross‐validated them to obtain the predictive performance for all combinations. By interpolating these outcomes, we derived continuous performance surfaces that reveal how model reliability changes along the gradient. Our analysis shows that process‐based models retain broader validity when calibrated in colder environments, whereas machine learning models exhibit more consistent but narrower validity. These validity domains demonstrate that both model structure and the calibration environment fundamentally influence transferability, offering a framework for identifying where predictions remain reliable under changing climate conditions.

Figures