Stacked adaptive dynamic programming with unknown system model

TL;DR

This paper introduces a novel adaptive dynamic programming method for optimal control of systems with unknown dynamics, utilizing a Kalman filter for model estimation and a stacked control scheme to enhance performance, validated through simulations.

Contribution

It proposes a new approach combining Kalman filter-based system modeling with stacked control to improve adaptive dynamic programming for unknown systems.

Findings

Enhanced control performance in simulations

Effective system modeling with Kalman filter

Outperforms baseline gradient descent method

Abstract

Adaptive dynamic programming is a collective term for a variety of approaches to infinite-horizon optimal control. Common to all approaches is approximation of the infinite-horizon cost function based on dynamic programming philosophy. Typically, they also require knowledge of a dynamical model of the system. In the current work, application of adaptive dynamic programming to a system whose dynamical model is unknown to the controller is addressed. In order to realize the control algorithm, a model of the system dynamics is estimated with a Kalman filter. A stacked control scheme to boost the controller performance is suggested. The functioning of the new approach was verified in simulation and compared to the baseline represented by gradient descent on the running cost.