Differential Set Selection via Confidence-Guided Entropy Minimization

TL;DR

This paper introduces a confidence-guided entropy minimization method for selecting minimal, influential variable subsets in binary classification, effectively handling finite-sample uncertainty and computational complexity.

Contribution

It presents an efficient iterative approach that uses confidence bounds to select variables reducing conditional entropy, addressing the NP-complete variable selection problem.

Findings

Successfully identifies influential variables in simulated data

Minimizes spurious selections with small samples

Offers a computationally tractable solution

Abstract

This paper addresses the challenge of identifying a minimal subset of discrete, independent variables that best predicts a binary class. We propose an efficient iterative method that sequentially selects variables based on which one provides the most statistically significant reduction in conditional entropy, using confidence bounds to account for finite-sample uncertainty. Tests on simulated data demonstrate the method's ability to correctly identify influential variables while minimizing spurious selections, even with small sample sizes, offering a computationally tractable solution to this NP-complete problem.