Penalized Likelihood Regression in Reproducing Kernel Hilbert Spaces with Randomized Covariate Data

TL;DR

This paper develops a penalized likelihood regression framework within reproducing kernel Hilbert spaces for situations where covariate data is incomplete or uncertain, providing theoretical guarantees and practical algorithms.

Contribution

It introduces a novel approach for penalized likelihood regression with randomized covariate data, including existence proof, dimension reduction, and smoothing parameter selection methods.

Findings

Existence of the penalized likelihood estimate under mild conditions

A dimension reduction technique for computation

A GACV method for smoothing parameter selection

Abstract

Classical penalized likelihood regression problems deal with the case that the independent variables data are known exactly. In practice, however, it is common to observe data with incomplete covariate information. We are concerned with a fundamentally important case where some of the observations do not represent the exact covariate information, but only a probability distribution. In this case, the maximum penalized likelihood method can be still applied to estimating the regression function. We first show that the maximum penalized likelihood estimate exists under a mild condition. In the computation, we propose a dimension reduction technique to minimize the penalized likelihood and derive a GACV (Generalized Approximate Cross Validation) to choose the smoothing parameter. Our methods are extended to handle more complicated incomplete data problems, such as, covariate measurement error and partially missing covariates.