Determinantal Point Processes Implicitly Regularize Semi-parametric Regression Problems

TL;DR

This paper explores how determinantal point processes (DPPs) implicitly regularize semi-parametric regression, providing theoretical insights and bounds on approximation risk, extending kernel ridge regression results.

Contribution

It introduces a formalism for finite DPPs in semi-parametric models, deriving an identity for implicit regularization and proposing a new projected Nyström approximation.

Findings

DPP sampling implicitly regularizes semi-parametric regression.

A novel projected Nyström approximation is introduced.

Bound on the expected risk of the approximation is derived.

Abstract

Semi-parametric regression models are used in several applications which require comprehensibility without sacrificing accuracy. Typical examples are spline interpolation in geophysics, or non-linear time series problems, where the system includes a linear and non-linear component. We discuss here the use of a finite Determinantal Point Process (DPP) for approximating semi-parametric models. Recently, Barthelm\'e, Tremblay, Usevich, and Amblard introduced a novel representation of some finite DPPs. These authors formulated extended L-ensembles that can conveniently represent partial-projection DPPs and suggest their use for optimal interpolation. With the help of this formalism, we derive a key identity illustrating the implicit regularization effect of determinantal sampling for semi-parametric regression and interpolation. Also, a novel projected Nystr\"om approximation is defined and used to derive a bound on the expected risk for the corresponding approximation of semi-parametric regression. This work naturally extends similar results obtained for kernel ridge regression.