Stochastic models for online optimization

TL;DR

This paper introduces control-theoretic algorithms, inspired by Kalman filtering and robust control, to improve online optimization of stochastic, dynamic quadratic functions, outperforming traditional gradient methods.

Contribution

It presents two novel algorithms tailored for stochastic linear models in online optimization, integrating Kalman filtering and $ ext{H}_ ext{infty}$ control techniques.

Findings

Algorithms outperform traditional gradient-based methods.

Significant performance gains over deterministic model-based strategies.

Effective handling of noise and uncertainty in dynamic environments.

Abstract

In this paper, we propose control-theoretic methods as tools for the design of online optimization algorithms that are able to address dynamic, noisy, and partially uncertain time-varying quadratic objective functions. Our approach introduces two algorithms specifically tailored for scenarios where the cost function follows a stochastic linear model. The first algorithm is based on a Kalman filter-inspired approach, leveraging state estimation techniques to account for the presence of noise in the evolution of the objective function. The second algorithm applies $\mathcal{H}_\infty$-robust control strategies to enhance performance under uncertainty, particularly in cases in which model parameters are characterized by a high variability. Through numerical experiments, we demonstrate that our algorithms offer significant performance advantages over the traditional gradient-based method and also over the optimization strategy proposed in arXiv:2205.13932 based on deterministic models.