Minimax Optimal Online Stochastic Learning for Sequences of Convex Functions under Sub-Gradient Observation Failures

TL;DR

This paper develops adaptive online convex optimization algorithms that achieve minimax optimal regret bounds even with stochastic sub-gradient observation failures, including noisy or missing data, and introduces a blind method for unknown stochastic properties.

Contribution

It proposes new algorithms with minimax optimal regret guarantees under stochastic sub-gradient observation faults, and a blind approach for unknown stochastic settings.

Findings

Algorithms achieve tight minimax optimal regret bounds.

The empirical method effectively combines stochastic gradient descent and adversarial algorithms.

Experimental results demonstrate robustness in various stochastic and adversarial scenarios.

Abstract

We study online convex optimization under stochastic sub-gradient observation faults, where we introduce adaptive algorithms with minimax optimal regret guarantees. We specifically study scenarios where our sub-gradient observations can be noisy or even completely missing in a stochastic manner. To this end, we propose algorithms based on sub-gradient descent method, which achieve tight minimax optimal regret bounds. When necessary, these algorithms utilize properties of the underlying stochastic settings to optimize their learning rates (step sizes). These optimizations are the main factor in providing the minimax optimal performance guarantees, especially when observations are stochastically missing. However, in real world scenarios, these properties of the underlying stochastic settings may not be revealed to the optimizer. For such a scenario, we propose a blind algorithm that estimates these properties empirically in a generally applicable manner. Through extensive experiments, we show that this empirical approach is a natural combination of regular stochastic gradient descent and the minimax optimal algorithms (which work best for randomized and adversarial function sequences, respectively).