On-line Prediction with Kernels and the Complexity Approximation Principle

TL;DR

This paper introduces an online kernel-based regression algorithm that leverages the Aggregating Algorithm and the Complexity Approximation Principle to achieve near-optimal performance, extending previous linear regression results.

Contribution

It generalizes linear regression results to kernel methods and develops an online algorithm with performance guarantees based on the Complexity Approximation Principle.

Findings

Algorithm performs nearly as well as any oblivious kernel predictor

Provides performance estimates for the kernel-based regression algorithm

Demonstrates the application of the Complexity Approximation Principle to kernel methods

Abstract

The paper describes an application of Aggregating Algorithm to the problem of regression. It generalizes earlier results concerned with plain linear regression to kernel techniques and presents an on-line algorithm which performs nearly as well as any oblivious kernel predictor. The paper contains the derivation of an estimate on the performance of this algorithm. The estimate is then used to derive an application of the Complexity Approximation Principle to kernel methods.