Adaptive tempering schedules with approximative intermediate measures for filtering problems

TL;DR

This paper introduces an adaptive tempering schedule for hybrid filtering methods in data assimilation, aiming to improve the stability and accuracy of ensemble and particle filters in high-dimensional, nonlinear systems.

Contribution

It proposes a novel adaptive tempering approach to optimize filter hybridization, addressing hyperparameter tuning challenges and enhancing filter performance.

Findings

Demonstrates improved filter stability in toy examples

Shows potential for better uncertainty quantification

Highlights the importance of adaptive tempering in complex systems

Abstract

Data assimilation algorithms integrate prior information from numerical model simulations with observed data. Ensemble-based filters, regarded as state-of-the-art, are widely employed for large-scale estimation tasks in disciplines such as geoscience and meteorology. Despite their inability to produce the true posterior distribution for nonlinear systems, their robustness and capacity for state tracking are noteworthy. In contrast, Particle filters yield the correct distribution in the ensemble limit but require substantially larger ensemble sizes than ensemble-based filters to maintain stability in higher-dimensional spaces. It is essential to transcend traditional Gaussian assumptions to achieve realistic quantification of uncertainties. One approach involves the hybridisation of filters, facilitated by tempering, to harness the complementary strengths of different filters. A new adaptive tempering method is proposed to tune the underlying schedule, aiming to systematically surpass the performance previously achieved. Although promising numerical results for certain filter combinations in toy examples exist in the literature, the tuning of hyperparameters presents a considerable challenge. A deeper understanding of these interactions is crucial for practical applications.