Understand Dynamic Regret with Switching Cost for Online Decision Making

TL;DR

This paper explores how switching costs influence dynamic regret in online decision making, revealing that switching costs significantly affect online algorithms but not in online convex optimization, supported by new theoretical analysis.

Contribution

It introduces a new framework analyzing the relation between switching costs and dynamic regret, showing different impacts in OA and OCO settings, and establishes a lower bound in OCO.

Findings

Switching costs impact dynamic regret in OA but not in OCO.

In OCO, switching costs do not alter the lower bound of regret.

Theoretical analysis differentiates the effect of switching costs across online decision frameworks.

Abstract

As a metric to measure the performance of an online method, dynamic regret with switching cost has drawn much attention for online decision making problems. Although the sublinear regret has been provided in many previous researches, we still have little knowledge about the relation between the dynamic regret and the switching cost. In the paper, we investigate the relation for two classic online settings: Online Algorithms (OA) and Online Convex Optimization (OCO). We provide a new theoretical analysis framework, which shows an interesting observation, that is, the relation between the switching cost and the dynamic regret is different for settings of OA and OCO. Specifically, the switching cost has significant impact on the dynamic regret in the setting of OA. But, it does not have an impact on the dynamic regret in the setting of OCO. Furthermore, we provide a lower bound of regret for the setting of OCO, which is same with the lower bound in the case of no switching cost. It shows that the switching cost does not change the difficulty of online decision making problems in the setting of OCO.