Estimates on Learning Rates for Multi-Penalty Distribution Regression

TL;DR

This paper introduces a novel multi-penalty regularization algorithm for distribution regression, providing optimal learning rates and extending analysis to nonstandard settings, with applications to distributed learning for large-scale data.

Contribution

The paper presents a new multi-penalty regularization algorithm for distribution regression and derives optimal learning rates, including for nonstandard cases where the target function is outside the RKHS.

Findings

Derived optimal learning rates for the proposed algorithm.

Extended analysis to nonstandard setting $f_{\rho}\notin\mathcal{H}_K$.

Proposed a distributed learning algorithm with optimal rates.

Abstract

This paper is concerned with functional learning by utilizing two-stage sampled distribution regression. We study a multi-penalty regularization algorithm for distribution regression under the framework of learning theory. The algorithm aims at regressing to real valued outputs from probability measures. The theoretical analysis on distribution regression is far from maturity and quite challenging, since only second stage samples are observable in practical setting. In the algorithm, to transform information from samples, we embed the distributions to a reproducing kernel Hilbert space $\mathcal{H}_K$ associated with Mercer kernel $K$ via mean embedding technique. The main contribution of the paper is to present a novel multi-penalty regularization algorithm to capture more features of distribution regression and derive optimal learning rates for the algorithm. The work also derives learning rates for distribution regression in the nonstandard setting $f_{\rho}\notin\mathcal{H}_K$, which is not explored in existing literature. Moreover, we propose a distribution regression-based distributed learning algorithm to face large-scale data or information challenge. The optimal learning rates are derived for the distributed learning algorithm. By providing new algorithms and showing their learning rates, we improve the existing work in different aspects in the literature.