Design and Experimental Test of Datatic Approximate Optimal Filter in Nonlinear Dynamic Systems

TL;DR

This paper introduces a novel data-driven filter called DAOF for nonlinear systems with non-Gaussian noise, leveraging reinforcement learning to improve estimation accuracy and efficiency, applicable with or without explicit system models.

Contribution

The paper proposes the DAOF framework, integrating reinforcement learning into nonlinear filtering, with two variants for systems with or without explicit models, and demonstrates its effectiveness through experimental validation.

Findings

DAOF-v1 outperforms existing nonlinear filters in accuracy and efficiency.

DAOF-v2 effectively filters without explicit system models.

Experimental results on vehicle systems validate the proposed methods.

Abstract

Filtering is crucial in engineering fields, providing vital state estimation for control systems. However, the nonlinear nature of complex systems and the presence of non-Gaussian noises pose significant challenges to the performance of conventional filtering methods in terms of estimation accuracy and computational efficiency. In this work, we present a data-driven closed-loop filter, termed datatic approximate optimal filter (DAOF), specifically designed for nonlinear systems under non-Gaussian conditions. We first formulate a Markovian filtering problem (MFP), which inherently shares a connection with reinforcement learning (RL) as it aims to compute the optimal state estimate by minimizing the accumulated error. To solve MFP, we propose DAOF, which primarily incorporates a trained RL policy and features two distinct structural designs: DAOF-v1 and DAOF-v2. Designed for systems with explicit models, DAOF-v1 combines prediction and update phases, with the RL policy generating the update value. Meanwhile, DAOF-v2 bypasses system modeling by directly outputting the state estimate. Then, we utilize an actor-critic algorithm to learn the parameterized policy for DAOF. Experimental results on a 2-degree-of-freedom (2-DOF) vehicle system, equipped with explicit system models, demonstrate the superior accuracy and computational efficiency of DAOF-v1 compared to existing nonlinear filters. Moreover, DAOF-v2 showcases its unique ability to perform filtering without requiring explicit system modeling, as validated by a 14-DOF vehicle system.