FSMJ: Feature Selection with Maximum Jensen-Shannon Divergence for Text Categorization

TL;DR

This paper introduces FSMJ, a new wrapper feature selection method based on Jensen-Shannon divergence for text categorization, utilizing real-valued features to improve discrimination and outperform existing methods.

Contribution

The paper proposes FSMJ, a novel greedy feature selection approach using JS-divergence with real-valued features, demonstrating superior performance over state-of-the-art methods.

Findings

FSMJ outperforms existing feature selection methods in text categorization.

JS-divergence increases monotonically with feature selection.

Real-valued features enhance discrimination in text classification.

Abstract

In this paper, we present a new wrapper feature selection approach based on Jensen-Shannon (JS) divergence, termed feature selection with maximum JS-divergence (FSMJ), for text categorization. Unlike most existing feature selection approaches, the proposed FSMJ approach is based on real-valued features which provide more information for discrimination than binary-valued features used in conventional approaches. We show that the FSMJ is a greedy approach and the JS-divergence monotonically increases when more features are selected. We conduct several experiments on real-life data sets, compared with the state-of-the-art feature selection approaches for text categorization. The superior performance of the proposed FSMJ approach demonstrates its effectiveness and further indicates its wide potential applications on data mining.