Empirical correction of a toy climate model

TL;DR

This paper investigates an empirical correction method for a toy climate model that improves forecast accuracy by reducing errors through a black-box approach, applicable to different model structures, but may alter the system dynamics.

Contribution

It introduces a model-agnostic empirical correction technique that enhances forecast accuracy and discusses methods to mitigate structural discrepancies caused by correction.

Findings

Correction improves forecast accuracy more than parameter tuning.

Structural differences between models and true system remain after correction.

A mitigation method reduces structural damage from empirical correction.

Abstract

Improving the accuracy of forecast models for physical systems such as the atmosphere is a crucial ongoing effort. Errors in state estimation for these often highly nonlinear systems has been the primary focus of recent research, but as that error has been successfully diminished, the role of model error in forecast uncertainty has duly increased. The present study is an investigation of a particular empirical correction procedure that is of special interest because it considers the model a "black box", and therefore can be applied widely with little modification. The procedure involves the comparison of short model forecasts with a reference "truth" system during a training period in order to calculate systematic (1) state-independent model bias and (2) state-dependent error patterns. An estimate of the likelihood of the latter error component is computed from the current state at every timestep of model integration. The effectiveness of this technique is explored in two experiments: (1) a perfect model scenario, in which models have the same structure and dynamics as the true system, differing only in parameter values; and (2) a more realistic scenario, in which models are structurally different (in dynamics, dimension, and parameterization) from the target system. In each case, the results suggest that the correction procedure is more effective for reducing error and prolonging forecast usefulness than parameter tuning. However, the cost of this increase in average forecast accuracy is the creation of substantial qualitative differences between the dynamics of the corrected model and the true system. A method to mitigate the structural damage caused by empirical correction and further increase forecast accuracy is presented.