Screening Methods for Classification Based on Non-parametric Bayesian Tests

TL;DR

This paper introduces a Bayesian-motivated variable screening method for classification tasks, demonstrating its effectiveness on simulated and real data, and proposing a kernel density-based classifier for improved accuracy.

Contribution

It proposes a novel Bayesian-inspired screening method for variable selection and compares it with existing techniques, also introducing a kernel density classifier for better classification performance.

Findings

The Bayesian screening method improves classification rates.

The method performs well on both simulated and real datasets.

Kernel density classifier can significantly outperform DART in some cases.

Abstract

Feature or variable selection is a problem inherent to large data sets. While many methods have been proposed to deal with this problem, some can scale poorly with the number of predictors in a data set. Screening methods scale linearly with the number of predictors by checking each predictor one at a time, and are a tool used to decrease the number of variables to consider before further analysis or variable selection. For classification, there is a variety of techniques. There are parametric based screening tests, such as t-test or SIS based screening, and non-parametric based screening tests, such as Kolmogorov distance based screening, and MV-SIS. We propose a method for variable screening that uses Bayesian-motivated tests, compare it to SIS based screening, and provide example applications of the method on simulated and real data. It is shown that our screening method can lead to improvements in classification rate. This is so even when our method is used in conjunction with a classifier, such as DART, which is designed to select a sparse subset of variables. Finally, we propose a classifier based on kernel density estimates that in some cases can produce dramatic improvements in classification rates relative to DART.