From Many to One: Consensus Inference in a MIP

TL;DR

This paper introduces an ANOVA-based method for deriving a weighted consensus estimate in Model Intercomparison Projects, improving the accuracy of combined scientific predictions by accounting for variability among models.

Contribution

It proposes a novel application of ANOVA models to weight model outputs in MIPs, enhancing consensus inference with variance-based weighting and Bayesian integration.

Findings

ANOVA weighting reduces the variance of the consensus estimate.

Application to CO2 flux MIP demonstrates improved inference accuracy.

Weighted consensus intervals are tighter than simple averages.

Abstract

A Model Intercomparison Project (MIP) consists of teams who each estimate the same underlying quantity (e.g., temperature projections to the year 2070), and the spread of the estimates indicates their uncertainty. It recognizes that a community of scientists will not agree completely but that there is value in looking for a consensus and information in the range of disagreement. A simple average of the teams' outputs gives a consensus estimate, but it does not recognize that some outputs are more variable than others. Statistical analysis of variance (ANOVA) models offer a way to obtain a weighted consensus estimate of outputs with a variance that is the smallest possible and hence the tightest possible 'one-sigma' and 'two-sigma' intervals. Modulo dependence between MIP outputs, the ANOVA approach weights a team's output inversely proportional to its variation. When external verification data are available for evaluating the fidelity of each MIP output, ANOVA weights can also provide a prior distribution for Bayesian Model Averaging to yield a consensus estimate. We use a MIP of carbon dioxide flux inversions to illustrate the ANOVA-based weighting and subsequent consensus inferences.