Extended Kalman filter -- Koopman operator for tractable stochastic optimal control

TL;DR

This paper introduces a novel approach using Koopman operator theory to reformulate stochastic optimal control problems, making them computationally tractable and solvable via standard LQR methods, especially for systems with varying observability.

Contribution

It presents a new reformulation of stochastic optimal control leveraging Koopman operators, enabling efficient solutions with dual control insights.

Findings

Effective control with varying observability demonstrated

Outperforms certainty equivalence control in numerical tests

Reformulates dual control as a standard LQR problem

Abstract

The theory of dual control was introduced more than seven decades ago. Although it has provided rich insights to the fields of control, estimation, and system identification, dual control is generally computationally prohibitive. In recent years, however, the use of Koopman operator theory for control applications has been emerging. This paper presents a new reformulation of the stochastic optimal control problem that, employing the Koopman operator, yields a standard LQR problem with the dual control as its solution. We provide a numerical example that demonstrates the effectiveness of the proposed approach compared with certainty equivalence control, when applied to systems with varying observability.