Minimax Adaptive Online Nonparametric Regression over Besov Spaces

TL;DR

This paper develops an adaptive wavelet-based online regression algorithm that achieves minimax-optimal regret bounds over Besov spaces, effectively handling spatially inhomogeneous smoothness without prior knowledge of parameters.

Contribution

It introduces a novel adaptive wavelet-based method for online nonparametric regression over Besov spaces, with dynamic adjustment to local regularity and optimal regret guarantees.

Findings

Achieves minimax-optimal regret bounds in Besov spaces.

Provides a locally adaptive extension for inhomogeneous environments.

Outperforms standard global methods in heterogeneous settings.

Abstract

We study online adversarial regression with convex losses against a rich class of continuous yet highly irregular prediction rules, modeled by Besov spaces $B\_{pq}^s$ with general parameters $1 \leq p,q \leq \infty$ and smoothness $s > \tfrac{d}{p}$. We introduce an adaptive wavelet-based algorithm that performs sequential prediction without prior knowledge of $(s,p,q)$, and establish minimax-optimal regret bounds against any comparator in $B\_{pq}^s$. We further design a locally adaptive extension capable of dynamically tracking spatially inhomogeneous smoothness. This adaptive mechanism adjusts the resolution of the predictions over both time and space, yielding refined regret bounds in terms of local regularity. Consequently, in heterogeneous environments, our adaptive guarantees can significantly surpass those obtained by standard global methods.