Spatially Adaptive Online Prediction of Piecewise Regular Functions

TL;DR

This paper introduces an online learning algorithm for estimating piecewise regular functions, providing strong risk bounds across all local regions, with efficient computation and improved guarantees over existing batch methods.

Contribution

It develops a modified sleeping experts aggregation algorithm with oracle risk bounds for all local regions, applicable to piecewise polynomial and bounded variation functions.

Findings

Achieves near-linear time computation in sample size.

Provides improved risk guarantees over existing batch estimators.

Demonstrates effectiveness in estimating piecewise polynomial functions.

Abstract

We consider the problem of estimating piecewise regular functions in an online setting, i.e., the data arrive sequentially and at any round our task is to predict the value of the true function at the next revealed point using the available data from past predictions. We propose a suitably modified version of a recently developed online learning algorithm called the sleeping experts aggregation algorithm. We show that this estimator satisfies oracle risk bounds simultaneously for all local regions of the domain. As concrete instantiations of the expert aggregation algorithm proposed here, we study an online mean aggregation and an online linear regression aggregation algorithm where experts correspond to the set of dyadic subrectangles of the domain. The resulting algorithms are near linear time computable in the sample size. We specifically focus on the performance of these online algorithms in the context of estimating piecewise polynomial and bounded variation function classes in the fixed design setup. The simultaneous oracle risk bounds we obtain for these estimators in this context provide new and improved (in certain aspects) guarantees even in the batch setting and are not available for the state of the art batch learning estimators.