Lepskii Principle in Supervised Learning

TL;DR

This paper introduces an adaptive, data-dependent regularization parameter selection rule for supervised learning with kernel methods, optimizing both prediction and reconstruction errors by leveraging a modified Lepskii balancing principle.

Contribution

It proposes a novel Lepskii-based method for selecting regularization parameters that adapt to unknown function regularity in kernel-based supervised learning.

Findings

Achieves optimal prediction and reconstruction errors

Adapts to unknown regularity of target functions

Provides a theoretically justified parameter selection rule

Abstract

In the setting of supervised learning using reproducing kernel methods, we propose a data-dependent regularization parameter selection rule that is adaptive to the unknown regularity of the target function and is optimal both for the least-square (prediction) error and for the reproducing kernel Hilbert space (reconstruction) norm error. It is based on a modified Lepskii balancing principle using a varying family of norms.