Online Learning in Dynamically Changing Environments

TL;DR

This paper investigates online learning in non-stationary environments, providing tight regret bounds that depend on the number of process changes and the complexity of the hypothesis class, advancing understanding of learning under changing data distributions.

Contribution

It introduces a novel framework for analyzing online learning with non-stationary data, establishing tight regret bounds that depend on process change complexity and hypothesis class VC dimension.

Findings

Established tight regret bounds for non-stationary processes with bounded changes.

Extended results to general mixable losses with improved bounds.

Demonstrated sub-linear regret for smooth adversary processes with threshold functions.

Abstract

We study the problem of online learning and online regret minimization when samples are drawn from a general unknown non-stationary process. We introduce the concept of a dynamic changing process with cost $K$, where the conditional marginals of the process can vary arbitrarily, but that the number of different conditional marginals is bounded by $K$ over $T$ rounds. For such processes we prove a tight (upto $\sqrt{\log T}$ factor) bound $O(\sqrt{KT\cdot\mathsf{VC}(\mathcal{H})\log T})$ for the expected worst case regret of any finite VC-dimensional class $\mathcal{H}$ under absolute loss (i.e., the expected miss-classification loss). We then improve this bound for general mixable losses, by establishing a tight (up to $\log^3 T$ factor) regret bound $O(K\cdot\mathsf{VC}(\mathcal{H})\log^3 T)$. We extend these results to general smooth adversary processes with unknown reference measure by showing a sub-linear regret bound for $1$-dimensional threshold functions under a general bounded convex loss. Our results can be viewed as a first step towards regret analysis with non-stationary samples in the distribution blind (universal) regime. This also brings a new viewpoint that shifts the study of complexity of the hypothesis classes to the study of the complexity of processes generating data.