Lepskii Principle for Distributed Kernel Ridge Regression

TL;DR

This paper introduces an adaptive distributed kernel ridge regression method using the Lepskii principle, addressing the challenge of parameter selection without data communication and aligning theoretical analysis with practical application.

Contribution

It develops Lep-AdaDKRR, an adaptive distributed kernel ridge regression method based on the Lepskii principle, with proven optimal learning rates and adaptability to various function regularities.

Findings

Achieves optimal learning rates for Lep-AdaDKRR.

Successfully adapts to regularity of functions and kernel decay rates.

Bridges the gap between theoretical analysis and practical application.

Abstract

Parameter selection without communicating local data is quite challenging in distributed learning, exhibing an inconsistency between theoretical analysis and practical application of it in tackling distributively stored data. Motivated by the recently developed Lepskii principle and non-privacy communication protocol for kernel learning, we propose a Lepskii principle to equip distributed kernel ridge regression (DKRR) and consequently develop an adaptive DKRR with Lepskii principle (Lep-AdaDKRR for short) by using a double weighted averaging synthesization scheme. We deduce optimal learning rates for Lep-AdaDKRR and theoretically show that Lep-AdaDKRR succeeds in adapting to the regularity of regression functions, effective dimension decaying rate of kernels and different metrics of generalization, which fills the gap of the mentioned inconsistency between theory and application.