Learning Adaptive Parameter Policies for Nonlinear Bayesian Filtering

TL;DR

This paper introduces reinforcement learning-based adaptive parameter policies for nonlinear Bayesian filters, improving estimate accuracy and robustness by dynamically adjusting parameters during filtering.

Contribution

It formulates adaptive parameter selection as a sequential decision problem and applies reinforcement learning to optimize parameter policies in nonlinear Bayesian filtering.

Findings

Learned policies enhance estimate quality.

Adaptive policies improve filter consistency.

Experiments show better performance with learned policies.

Abstract

For many nonlinear Bayesian state estimation problems, the posterior recursion is not analytically tractable, leading to algorithms that are influenced by numerical approximation errors. These algorithms depend on parameters that affect the approximation's accuracy and computational cost. The parameters include, for example, the number of particles, scaling parameters, and the number of iterations in iterative computations. Typically, these parameters are fixed or adjusted heuristically, although the approximation accuracy can change over time with the local degree of nonlinearity and uncertainty. The approximation errors introduced at a time step propagate through subsequent updates, affecting the accuracy, consistency, and robustness of future estimates. This paper presents adaptive parameter selection in nonlinear Bayesian filtering as a sequential decision-making problem, where parameters influence not only the immediate estimation outcome but also the future estimates. The decision-making problem is addressed using reinforcement learning to learn adaptive parameter policies for nonlinear Bayesian filters. Experiments with the unscented Kalman filter and stochastic integration filter demonstrate that the learned policies improve both estimate quality and consistency.