Regret-Guaranteed Safe Switching with Minimum Cost: LQR Setting with Unknown Dynamics

TL;DR

This paper introduces a regret-minimizing control strategy for externally forced switched systems with unknown dynamics, ensuring safety through online learning of dwell times and achieving sublinear regret.

Contribution

It develops an online algorithm based on optimism in the face of uncertainty to learn dwell times and control policies for safe, cost-effective switching with unknown system parameters.

Findings

Expected regret of (|M|   ns)

Effective online estimation of dwell times

Benchmarking shows improved performance over known-parameter scenarios

Abstract

Externally Forced Switched (EFS) systems represent a subset of switched systems where switches occur deliberately to meet an external requirement. However, fast switching can lead to instability, even when all closed-loop modes are stable. In this study, our focus is on an EFS scenario with \textit{unknown system dynamics}, where the next mode to switch to is revealed by an external entity in real-time as the switch occurs. The challenge is to track the revealed sequence while (1) minimizing accumulated cost in a regretful sense and (2) ensuring that the norm of the system's state does not grow excessively-a property we refer to as 'the safety of switching.' Achieving the latter involves requiring the closed-loop system to remain in each revealed mode for some minimum dwell time, which must be learned online. We propose an algorithm based on the principles of Optimism in the Face of Uncertainty. This algorithm jointly establishes confidence sets for unknown parameters, devises a feedback policy, and estimates a minimum dwell time for each revealed mode from data. By precisely estimating dwell-time error, our strategy yields an expected regret of $\mathcal{O}(|M| \sqrt{ns})$, where $ns$ and $|M|$ denote the total switches and mode count, respectively. We benchmark this approach against scenarios with known parameters.