Analysis of Regularized Least Squares in Reproducing Kernel Krein Spaces

TL;DR

This paper investigates the asymptotic behavior of regularized least squares with indefinite kernels in Reproducing Kernel Krein Spaces, providing theoretical insights and convergence rates comparable to those in RKHS.

Contribution

It introduces a bounded hyper-sphere constraint to analyze the non-convex problem and derives the first approximation analysis of regularized learning in RKKS.

Findings

Global optimal solution with closed form on the sphere

Convergence results for hypothesis error

Learning rates comparable to RKHS

Abstract

In this paper, we study the asymptotic properties of regularized least squares with indefinite kernels in reproducing kernel Krein spaces (RKKS). By introducing a bounded hyper-sphere constraint to such non-convex regularized risk minimization problem, we theoretically demonstrate that this problem has a globally optimal solution with a closed form on the sphere, which makes approximation analysis feasible in RKKS. Regarding to the original regularizer induced by the indefinite inner product, we modify traditional error decomposition techniques, prove convergence results for the introduced hypothesis error based on matrix perturbation theory, and derive learning rates of such regularized regression problem in RKKS. Under some conditions, the derived learning rates in RKKS are the same as that in reproducing kernel Hilbert spaces (RKHS), which is actually the first work on approximation analysis of regularized learning algorithms in RKKS.