Learning to steer with Brownian noise

TL;DR

This paper introduces algorithms that combine statistical learning with stochastic control to address a bounded velocity follower problem, achieving logarithmic regret without prior system knowledge.

Contribution

It develops a novel framework integrating empirical averages with stochastic control, providing the first logarithmic regret guarantees for this ergodic control setting.

Findings

Achieved logarithmic expected regret rate.

Provided convergence analysis of ergodic processes.

Validated the effectiveness of the proposed algorithms.

Abstract

This paper considers an ergodic version of the bounded velocity follower problem, assuming that the decision maker lacks knowledge of the underlying system parameters and must learn them while simultaneously controlling. We propose algorithms based on moving empirical averages and develop a framework for integrating statistical methods with stochastic control theory. Our primary result is a logarithmic expected regret rate. To achieve this, we conduct a rigorous analysis of the ergodic convergence rates of the underlying processes and the risks of the considered estimators.