Causally-Aware Unsupervised Feature Selection Learning

TL;DR

This paper introduces CAUSE-FS, a novel unsupervised feature selection method that incorporates causal inference to improve feature relevance and interpretability in high-dimensional data.

Contribution

CAUSE-FS uniquely integrates causal regularization and hierarchical clustering to enhance feature selection by accounting for causal relationships and reducing spurious correlations.

Findings

Outperforms state-of-the-art UFS methods in various datasets

Improves interpretability through causal feature visualization

Effectively captures local data structure with adaptive similarity graphs

Abstract

Unsupervised feature selection (UFS) has recently gained attention for its effectiveness in processing unlabeled high-dimensional data. However, existing methods overlook the intrinsic causal mechanisms within the data, resulting in the selection of irrelevant features and poor interpretability. Additionally, previous graph-based methods fail to account for the differing impacts of non-causal and causal features in constructing the similarity graph, which leads to false links in the generated graph. To address these issues, a novel UFS method, called Causally-Aware UnSupErvised Feature Selection learning (CAUSE-FS), is proposed. CAUSE-FS introduces a novel causal regularizer that reweights samples to balance the confounding distribution of each treatment feature. This regularizer is subsequently integrated into a generalized unsupervised spectral regression model to mitigate spurious associations between features and clustering labels, thus achieving causal feature selection. Furthermore, CAUSE-FS employs causality-guided hierarchical clustering to partition features with varying causal contributions into multiple granularities. By integrating similarity graphs learned adaptively at different granularities, CAUSE-FS increases the importance of causal features when constructing the fused similarity graph to capture the reliable local structure of data. Extensive experimental results demonstrate the superiority of CAUSE-FS over state-of-the-art methods, with its interpretability further validated through feature visualization.