Online Convex Optimization with Memory and Limited Predictions

TL;DR

This paper introduces a new online convex optimization algorithm that leverages limited future predictions and memory to achieve exponentially decaying regret, supported by strong theoretical guarantees and numerical validation.

Contribution

It presents a novel predictive algorithm for online convex optimization with memory and limited predictions, providing exponential regret decay and new subroutines with optimal convergence rates.

Findings

Dynamic regret decays exponentially with prediction window length.

Achieves $ ext{sqrt}(TV_T)$-dynamic regret with bandit feedback.

Attains linear convergence rate for convex optimization.

Abstract

This paper addresses an online convex optimization problem where the cost function at each step depends on a history of past decisions (i.e., memory), and the decision maker has access to limited predictions of future cost values within a finite window. The goal is to design an algorithm that minimizes the dynamic regret against the optimal sequence of decisions in hindsight. To this end, we propose a novel predictive algorithm and establish strong theoretical guarantees for its performance. We show that the algorithm's dynamic regret decays exponentially with the length of the prediction window. Our algorithm comprises two general subroutines of independent interest. The first subroutine solves online convex optimization with memory and bandit feedback, achieving a $\sqrt{TV_T}$-dynamic regret, where $V_T$ measures the variation of the optimal decision sequence. The second is a zeroth-order method that attains a linear convergence rate for general convex optimization, matching the best achievable rate of first-order methods. The key to our algorithm is a novel truncated Gaussian smoothing technique when querying the decision points to obtain the predictions. We validate our theoretical results with numerical experiments.