Causal Discovery and Classification Using Lempel-Ziv Complexity

TL;DR

This paper introduces a new causality measure based on Lempel-Ziv complexity and demonstrates its effectiveness in decision trees for identifying causal features, outperforming traditional methods on causally structured data.

Contribution

The paper proposes a novel Lempel-Ziv complexity-based causality measure and integrates it into decision trees to improve causal inference capabilities.

Findings

Causality-based decision trees outperform traditional Gini-based trees on causal datasets.

The LZ causality measure effectively captures causal relationships in data.

A causal strength metric for features is derived from the LZ measure.

Abstract

Inferring causal relationships in the decision-making processes of machine learning algorithms is a crucial step toward achieving explainable Artificial Intelligence (AI). In this research, we introduce a novel causality measure and a distance metric derived from Lempel-Ziv (LZ) complexity. We explore how the proposed causality measure can be used in decision trees by enabling splits based on features that most strongly \textit{cause} the outcome. We further evaluate the effectiveness of the causality-based decision tree and the distance-based decision tree in comparison to a traditional decision tree using Gini impurity. While the proposed methods demonstrate comparable classification performance overall, the causality-based decision tree significantly outperforms both the distance-based decision tree and the Gini-based decision tree on datasets generated from causal models. This result indicates that the proposed approach can capture insights beyond those of classical decision trees, especially in causally structured data. Based on the features used in the LZ causal measure based decision tree, we introduce a causal strength for each features in the dataset so as to infer the predominant causal variables for the occurrence of the outcome.