Coefficient-based Regularized Distribution Regression

TL;DR

This paper introduces a novel coefficient-based regularized distribution regression method that handles indefinite kernels, providing optimal learning rates and a new paradigm for distribution regression with improved saturation effects.

Contribution

It presents the first distribution regression algorithm using indefinite kernels, with comprehensive asymptotic analysis and optimal learning rates under mild conditions.

Findings

Achieves optimal learning rates matching minimax bounds.

Handles indefinite kernels without requiring symmetry or positive semi-definiteness.

Improves upon existing methods by reducing saturation effects.

Abstract

In this paper, we consider the coefficient-based regularized distribution regression which aims to regress from probability measures to real-valued responses over a reproducing kernel Hilbert space (RKHS), where the regularization is put on the coefficients and kernels are assumed to be indefinite. The algorithm involves two stages of sampling, the first stage sample consists of distributions and the second stage sample is obtained from these distributions. Asymptotic behaviors of the algorithm in different regularity ranges of the regression function are comprehensively studied and learning rates are derived via integral operator techniques. We get the optimal rates under some mild conditions, which matches the one-stage sampled minimax optimal rate. Compared with the kernel methods for distribution regression in the literature, the algorithm under consideration does not require the kernel to be symmetric and positive semi-definite and hence provides a simple paradigm for designing indefinite kernel methods, which enriches the theme of the distribution regression. To the best of our knowledge, this is the first result for distribution regression with indefinite kernels, and our algorithm can improve the saturation effect.