Multivalued Subsets Under Information Theory

TL;DR

This paper explores enhancing decision tree algorithms by combining attribute-values to improve information gain, using a heuristic approach for better feature selection in data mining.

Contribution

It introduces a novel method for combining attribute-values to optimize information gain in decision trees, improving feature selection processes.

Findings

Improved information gain through combined attribute-values.

Heuristic method outperforms GID3 in feature selection.

Statistical analysis confirms effectiveness of the approach.

Abstract

In the fields of finance, engineering and sciences data mining/ machine learning has held an eminent position in predictive analysis. Complex algorithms and adaptive decision models have contributed towards streamlining research as well as improve forecasting. Extensive study in areas surrounding computation and mathematical sciences has primarily been responsible for the field's development. Classification based modeling, which holds a prominent position amongst the different rule-based algorithms, is one of the most widely used decision making tool. The decision tree has a place of profound significance in classification modeling. A number of heuristics have been developed over the years to refine its decision making process. Most heuristics applied to such tree-based learning algorithms derive their roots from Shannon's 'Information Theory'. The current application of this theory is directed towards individual assessment of the attribute-values. The proposed study takes a look at the effects of combining these values with the aim to improve the 'Information Gain'. A search-based heuristic tool is applied for identifying the subsets sharing a better gain value than the ones presented in the GID3 approach. An application towards the feature selection stage of the mining process has been tested and presented with statistical analysis.