Incorporating Causal Graphical Prior Knowledge into Predictive Modeling via Simple Data Augmentation

TL;DR

This paper introduces a simple, model-agnostic data augmentation technique that leverages causal graphical prior knowledge to improve predictive modeling, especially with limited data, by reducing overfitting and enhancing accuracy.

Contribution

It proposes a novel data augmentation method that incorporates causal graph-based conditional independence knowledge into supervised learning models.

Findings

Improves prediction accuracy in small-data scenarios.

Reduces overfitting by decreasing model complexity.

Effective across real-world datasets with domain-provided causal graphs.

Abstract

Causal graphs (CGs) are compact representations of the knowledge of the data generating processes behind the data distributions. When a CG is available, e.g., from the domain knowledge, we can infer the conditional independence (CI) relations that should hold in the data distribution. However, it is not straightforward how to incorporate this knowledge into predictive modeling. In this work, we propose a model-agnostic data augmentation method that allows us to exploit the prior knowledge of the CI encoded in a CG for supervised machine learning. We theoretically justify the proposed method by providing an excess risk bound indicating that the proposed method suppresses overfitting by reducing the apparent complexity of the predictor hypothesis class. Using real-world data with CGs provided by domain experts, we experimentally show that the proposed method is effective in improving the prediction accuracy, especially in the small-data regime.