Online Bandit Nonlinear Control with Dynamic Batch Length and Adaptive Learning Rate

TL;DR

This paper introduces the DBAR algorithm for online bandit nonlinear control, which adaptively learns stabilizing controllers with dynamic batch length and learning rate, achieving stability and low regret even with weaker stability assumptions.

Contribution

The paper proposes the DBAR algorithm that improves stability certification and regret bounds in online bandit nonlinear control by using dynamic batch length and adaptive learning rate.

Findings

DBAR achieves asymptotic stability with weaker controller stability assumptions.

The algorithm attains tight regret bounds using only state norm information.

DBAR outperforms existing methods requiring exponential stabilizability.

Abstract

This paper is concerned with the online bandit nonlinear control, which aims to learn the best stabilizing controller from a pool of stabilizing and destabilizing controllers of unknown types for a given nonlinear dynamical system. We develop an algorithm, named Dynamic Batch length and Adaptive learning Rate (DBAR), and study its stability and regret. Unlike the existing Exp3 algorithm requiring an exponentially stabilizing controller, DBAR only needs a significantly weaker notion of controller stability, in which case substantial time may be required to certify the system stability. Dynamic batch length in DBAR effectively addresses this issue and enables the system to attain asymptotic stability, where the algorithm behaves as if there were no destabilizing controllers. Moreover, adaptive learning rate in DBAR only uses the state norm information to achieve a tight regret bound even when none of the stabilizing controllers in the pool are exponentially stabilizing.