# Equipping Experts/Bandits with Long-term Memory

## TL;DR

This paper introduces a reduction-based method for achieving long-term memory guarantees in online learning, providing new algorithms with improved regret bounds and extending results to sparse bandit settings.

## Contribution

It presents the first reduction-based approach for long-term memory guarantees in online learning, achieving optimal regret bounds and extending to sparse multi-armed bandits.

## Key findings

- Developed algorithms with regret of order √T(S ln T + n ln K).
- Achieved simultaneous adaptation to stochastic and adversarial environments.
- Provided lower bounds showing sparse losses do not improve worst-case regret.

## Abstract

We propose the first reduction-based approach to obtaining long-term memory guarantees for online learning in the sense of Bousquet and Warmuth, 2002, by reducing the problem to achieving typical switching regret. Specifically, for the classical expert problem with $K$ actions and $T$ rounds, using our framework we develop various algorithms with a regret bound of order $\mathcal{O}(\sqrt{T(S\ln T + n \ln K)})$ compared to any sequence of experts with $S-1$ switches among $n \leq \min\{S, K\}$ distinct experts. In addition, by plugging specific adaptive algorithms into our framework we also achieve the best of both stochastic and adversarial environments simultaneously. This resolves an open problem of Warmuth and Koolen, 2014. Furthermore, we extend our results to the sparse multi-armed bandit setting and show both negative and positive results for long-term memory guarantees. As a side result, our lower bound also implies that sparse losses do not help improve the worst-case regret for contextual bandits, a sharp contrast with the non-contextual case.