Targeted Random Projection for Prediction from High-Dimensional Features

TL;DR

This paper introduces TARP, a targeted random projection method that combines screening and projection techniques for efficient prediction in high-dimensional, correlated data settings, with theoretical guarantees and practical advantages.

Contribution

The paper proposes a novel TARP method that integrates screening-informed feature inclusion probabilities into random projections, improving high-dimensional prediction performance.

Findings

TARP outperforms existing methods in simulated data.

TARP demonstrates improved accuracy on real datasets.

Theoretical guarantees support the Bayesian predictive algorithm.

Abstract

We consider the problem of computationally-efficient prediction from high dimensional and highly correlated predictors in challenging settings where accurate variable selection is effectively impossible. Direct application of penalization or Bayesian methods implemented with Markov chain Monte Carlo can be computationally daunting and unstable. Hence, some type of dimensionality reduction prior to statistical analysis is in order. Common solutions include application of screening algorithms to reduce the regressors, or dimension reduction using projections of the design matrix. The former approach can be highly sensitive to threshold choice in finite samples, while the later can have poor performance in very high-dimensional settings. We propose a TArgeted Random Projection (TARP) approach that combines positive aspects of both strategies to boost performance. In particular, we propose to use information from independent screening to order the inclusion probabilities of the features in the projection matrix used for dimension reduction, leading to data-informed sparsity. We provide theoretical support for a Bayesian predictive algorithm based on TARP, including both statistical and computational complexity guarantees. Examples for simulated and real data applications illustrate gains relative to a variety of competitors.