A Projection Approach to Local Regression with Variable-Dimension Covariates

TL;DR

This paper introduces a novel imputation-free local regression method for incomplete covariate data, capable of flexible predictions and uncertainty quantification, demonstrated through synthetic and real data comparisons.

Contribution

It develops a variable-dimension covariate model with analytical projection and efficient MCMC sampling, advancing prediction accuracy without imputation.

Findings

Effective nonlinear prediction demonstrated on synthetic data.

Outperforms existing methods in handling missing covariates.

Provides consistent uncertainty propagation in predictions.

Abstract

Incomplete covariate vectors are known to be problematic for estimation and inferences on model parameters, but their impact on prediction performance is less understood. We develop an imputation-free method that builds on a random partition model admitting variable-dimension covariates. Cluster-specific response models further incorporate covariates via linear predictors, facilitating estimation of smooth prediction surfaces with relatively few clusters. We exploit marginalization techniques of Gaussian kernels to analytically project response distributions according to any pattern of missing covariates, yielding a local regression with internally consistent uncertainty propagation that utilizes only one set of coefficients per cluster. Aggressive shrinkage of these coefficients regulates uncertainty due to missing covariates. The method allows in- and out-of-sample prediction for any missingness pattern, even if the pattern in a new subject's incomplete covariate vector was not seen in the training data. We develop an MCMC algorithm for posterior sampling that improves a computationally expensive update for latent cluster allocation. Finally, we demonstrate the model's effectiveness for nonlinear point and density prediction under various circumstances by comparing with other recent methods for regression of variable dimensions on synthetic and real data.