Kernel Feature Selection via Conditional Covariance Minimization

TL;DR

This paper introduces a kernel-based feature selection method that optimally identifies covariates most predictive of the response by minimizing conditional covariance, with proven consistency and competitive performance.

Contribution

It presents a novel kernel-based approach for feature selection using conditional covariance minimization, with theoretical guarantees and empirical validation.

Findings

Method outperforms existing algorithms on synthetic data.

Method shows competitive results on real datasets.

Proven consistency of the feature selection procedure.

Abstract

We propose a method for feature selection that employs kernel-based measures of independence to find a subset of covariates that is maximally predictive of the response. Building on past work in kernel dimension reduction, we show how to perform feature selection via a constrained optimization problem involving the trace of the conditional covariance operator. We prove various consistency results for this procedure, and also demonstrate that our method compares favorably with other state-of-the-art algorithms on a variety of synthetic and real data sets.