Abstraction-based branch and bound approach to Q-learning for hybrid optimal control

TL;DR

This paper introduces an abstraction-based branch and bound method for Q-learning tailored to hybrid optimal control, integrating approximate dynamic programming, alternating simulation, and Lagrangian duality to improve control policies.

Contribution

It develops a novel theoretical framework combining model predictive control, approximate dynamic programming, and branch and bound techniques for hybrid systems.

Findings

Successfully applied to a numerical example

Enhanced Q-function refinement using Lagrangian duality

Framework enables control of complex hybrid systems

Abstract

In this paper, we design a theoretical framework allowing to apply model predictive control on hybrid systems. For this, we develop a theory of approximate dynamic programming by leveraging the concept of alternating simulation. We show how to combine these notions in a branch and bound algorithm that can further refine the Q-functions using Lagrangian duality. We illustrate the approach on a numerical example.