Online Optimization with Feedback Delay and Nonlinear Switching Cost

TL;DR

This paper introduces a novel online optimization algorithm, iROBD, that effectively handles delayed feedback and nonlinear multi-step switching costs, achieving a constant competitive ratio independent of problem dimension.

Contribution

The paper proposes the iROBD algorithm with a constant, dimension-free competitive ratio for online optimization with feedback delay and nonlinear switching costs, supported by tight lower bounds.

Findings

iROBD achieves an $O(L^{2k})$ competitive ratio.

Lower bounds show the necessity of the Lipschitz condition and tightness of dependencies.

Reductions connect the problem to online control with delay and nonlinear dynamics.

Abstract

We study a variant of online optimization in which the learner receives $k$-round $\textit{delayed feedback}$ about hitting cost and there is a multi-step nonlinear switching cost, i.e., costs depend on multiple previous actions in a nonlinear manner. Our main result shows that a novel Iterative Regularized Online Balanced Descent (iROBD) algorithm has a constant, dimension-free competitive ratio that is $O(L^{2k})$, where $L$ is the Lipschitz constant of the switching cost. Additionally, we provide lower bounds that illustrate the Lipschitz condition is required and the dependencies on $k$ and $L$ are tight. Finally, via reductions, we show that this setting is closely related to online control problems with delay, nonlinear dynamics, and adversarial disturbances, where iROBD directly offers constant-competitive online policies.