Online nonparametric regression with Sobolev kernels

TL;DR

This paper analyzes the regret bounds of online kernelized ridge regression using Sobolev kernels in adversarial nonparametric settings, establishing near-optimal rates and comparing computational and statistical performance.

Contribution

It derives regret upper bounds for online Sobolev kernel regression and demonstrates their near-optimality through minimax analysis, also comparing with existing non-parametric methods.

Findings

Regret bounds are established for Sobolev kernel regression.

Rates are shown to be minimax optimal in certain cases.

Comparison with traditional non-parametric methods highlights computational and statistical advantages.

Abstract

In this work we investigate the variation of the online kernelized ridge regression algorithm in the setting of $d-$dimensional adversarial nonparametric regression. We derive the regret upper bounds on the classes of Sobolev spaces $W_{p}^{\beta}(\mathcal{X})$, $p\geq 2, \beta>\frac{d}{p}$. The upper bounds are supported by the minimax regret analysis, which reveals that in the cases $\beta> \frac{d}{2}$ or $p=\infty$ these rates are (essentially) optimal. Finally, we compare the performance of the kernelized ridge regression forecaster to the known non-parametric forecasters in terms of the regret rates and their computational complexity as well as to the excess risk rates in the setting of statistical (i.i.d.) nonparametric regression.