What is the correct cost functional for variational data assimilation?

TL;DR

This paper clarifies that the Onsager--Machlup functional, not the energy functional, correctly represents the most probable path in stochastic data assimilation, impacting practical modeling and approximation methods.

Contribution

It resolves confusion in the literature by demonstrating the Onsager--Machlup functional's correctness over the energy functional for MAP estimation in stochastic models.

Findings

The energy functional does not generally yield the most probable path.

Using the Onsager--Machlup functional improves approximation accuracy.

Implications for discrete-time schemes in practical data assimilation.

Abstract

Variational approaches to data assimilation, and weakly constrained four dimensional variation (WC-4DVar) in particular, are important in the geosciences but also in other communities (often under different names). The cost functions and the resulting optimal trajectories may have a probabilistic interpretation, for instance by linking data assimilation with Maximum Aposteriori (MAP) estimation. This is possible in particular if the unknown trajectory is modelled as the solution of a stochastic differential equation (SDE), as is increasingly the case in weather forecasting and climate modelling. In this case, the MAP estimator (or "most probable path" of the SDE) is obtained by minimising the Onsager--Machlup functional. Although this fact is well known, there seems to be some confusion in the literature, with the energy (or "least squares") functional sometimes been claimed to yield the most probable path. The first aim of this paper is to address this confusion and show that the energy functional does not, in general, provide the most probable path. The second aim is to discuss the implications in practice. Although the mentioned results pertain to stochastic models in continuous time, they do have consequences in practice where SDE's are approximated by discrete time schemes. It turns out that using an approximation to the SDE and calculating its most probable path does not necessarily yield a good approximation to the most probable path of the SDE proper. This suggest that even in discrete time, a version of the Onsager--Machlup functional should be used, rather than the energy functional, at least if the solution is to be interpreted as a MAP estimator.