Feature Selection based on the Local Lift Dependence Scale

TL;DR

This paper introduces a novel feature selection method based on the Local Lift Dependence Scale, which analyzes local properties of joint distributions to improve feature relevance assessment.

Contribution

It extends the search space for feature selection to include feature-value pairs and applies a local mutual information measure for more detailed dependence analysis.

Findings

Effective feature selection on educational and UCI datasets.

Enhanced detection of feature-value influence on target variable.

Demonstrated advantages over classical global measures.

Abstract

This paper uses a classical approach to feature selection: minimization of a cost function applied on estimated joint distributions. However, the search space in which such minimization is performed is extended. In the original formulation, the search space is the Boolean lattice of features sets (BLFS), while, in the present formulation, it is a collection of Boolean lattices of ordered pairs (features, associated value) (CBLOP), indexed by the elements of the BLFS. In this approach, we may not only select the features that are most related to a variable Y, but also select the values of the features that most influence the variable or that are most prone to have a specific value of Y. A local formulation of Shanon's mutual information is applied on a CBLOP to select features, namely, the Local Lift Dependence Scale, an scale for measuring variable dependence in multiple resolutions. The main contribution of this paper is to define and apply this local measure, which permits to analyse local properties of joint distributions that are neglected by the classical Shanon's global measure. The proposed approach is applied to a dataset consisting of student performances on a university entrance exam, as well as on undergraduate courses. The approach is also applied to two datasets of the UCI Machine Learning Repository.