Enhancing Model Robustness and Fairness with Causality: A Regularization Approach

TL;DR

This paper introduces a causal regularization method to improve machine learning model robustness and fairness by emphasizing causal features and reducing reliance on spurious correlations, validated through experiments.

Contribution

It presents a novel regularization approach that incorporates causal knowledge into model training to enhance robustness and fairness, based on manually identified causal and spurious features.

Findings

Significant improvement in model robustness against counterfactual texts.

Enhanced fairness with respect to sensitive attributes.

Effective separation of causal and spurious features during training.

Abstract

Recent work has raised concerns on the risk of spurious correlations and unintended biases in statistical machine learning models that threaten model robustness and fairness. In this paper, we propose a simple and intuitive regularization approach to integrate causal knowledge during model training and build a robust and fair model by emphasizing causal features and de-emphasizing spurious features. Specifically, we first manually identify causal and spurious features with principles inspired from the counterfactual framework of causal inference. Then, we propose a regularization approach to penalize causal and spurious features separately. By adjusting the strength of the penalty for each type of feature, we build a predictive model that relies more on causal features and less on non-causal features. We conduct experiments to evaluate model robustness and fairness on three datasets with multiple metrics. Empirical results show that the new models built with causal awareness significantly improve model robustness with respect to counterfactual texts and model fairness with respect to sensitive attributes.