Minimax-optimal and Locally-adaptive Online Nonparametric Regression

TL;DR

This paper introduces a computationally efficient online nonparametric regression algorithm that adaptively achieves minimax optimal rates by exploiting local smoothness patterns without prior knowledge.

Contribution

It presents the first locally adaptive, minimax optimal online regression algorithm using chaining trees that dynamically adjusts to local H{"o}lder continuity.

Findings

Achieves minimax optimal regret bounds in adversarial settings.

Dynamically adapts to local smoothness variations.

Provides a foundation for future extensions to boosting frameworks.

Abstract

We study adversarial online nonparametric regression with general convex losses and propose a parameter-free learning algorithm that achieves minimax optimal rates. Our approach leverages chaining trees to compete against H{\"o}lder functions and establishes optimal regret bounds. While competing with nonparametric function classes can be challenging, they often exhibit local patterns - such as local H{\"o}lder continuity - that online algorithms can exploit. Without prior knowledge, our method dynamically tracks and adapts to different H{\"o}lder profiles by pruning a core chaining tree structure, aligning itself with local smoothness variations. This leads to the first computationally efficient algorithm with locally adaptive optimal rates for online regression in an adversarial setting. Finally, we discuss how these notions could be extended to a boosting framework, offering promising directions for future research.