Supervised Feature Selection via Dependence Estimation

TL;DR

This paper presents a new supervised feature selection method that uses the Hilbert-Schmidt Independence Criterion to measure dependence between features and labels, unifying various learning tasks.

Contribution

It introduces a dependence-based feature filtering framework applicable to classification and regression, with an efficient backward-elimination algorithm for approximation.

Findings

Effective on artificial datasets

Demonstrates usefulness on real-world datasets

Unifies feature selection across supervised tasks

Abstract

We introduce a framework for filtering features that employs the Hilbert-Schmidt Independence Criterion (HSIC) as a measure of dependence between the features and the labels. The key idea is that good features should maximise such dependence. Feature selection for various supervised learning problems (including classification and regression) is unified under this framework, and the solutions can be approximated using a backward-elimination algorithm. We demonstrate the usefulness of our method on both artificial and real world datasets.