Efficient kernel-based variable selection with sparsistency

TL;DR

This paper introduces a scalable, model-free kernel-based variable selection algorithm that guarantees asymptotic sparsistency and can be extended to interaction effects, suitable for high-dimensional data analysis.

Contribution

It develops a three-step, kernel-based variable selection method with theoretical guarantees, scalable computation, and broad applicability to various RKHS and predictor effects.

Findings

Algorithm achieves asymptotic sparsistency under mild conditions.

Computational cost is linear in data dimension, suitable for large datasets.

Effective in simulated and real data examples.

Abstract

Variable selection is central to high-dimensional data analysis, and various algorithms have been developed. Ideally, a variable selection algorithm shall be flexible, scalable, and with theoretical guarantee, yet most existing algorithms cannot attain these properties at the same time. In this article, a three-step variable selection algorithm is developed, involving kernel-based estimation of the regression function and its gradient functions as well as a hard thresholding. Its key advantage is that it assumes no explicit model assumption, admits general predictor effects, allows for scalable computation, and attains desirable asymptotic sparsistency. The proposed algorithm can be adapted to any reproducing kernel Hilbert space (RKHS) with different kernel functions, and can be extended to interaction selection with slight modification. Its computational cost is only linear in the data dimension, and can be further improved through parallel computing. The sparsistency of the proposed algorithm is established for general RKHS under mild conditions, including linear and Gaussian kernels as special cases. Its effectiveness is also supported by a variety of simulated and real examples.