Unifying Causal Reinforcement Learning: Survey, Taxonomy, Algorithms and Applications

TL;DR

This survey reviews recent advances in causal reinforcement learning, emphasizing how integrating causal inference improves robustness, explainability, and generalization in RL systems across various applications.

Contribution

It provides a comprehensive taxonomy of CRL approaches, analyzes current challenges, and outlines future directions for research in this emerging field.

Findings

Causal RL enhances robustness against distribution shifts.

Empirical successes demonstrate improved interpretability.

Structured categorization aids understanding of CRL methods.

Abstract

Integrating causal inference (CI) with reinforcement learning (RL) has emerged as a powerful paradigm to address critical limitations in classical RL, including low explainability, lack of robustness and generalization failures. Traditional RL techniques, which typically rely on correlation-driven decision-making, struggle when faced with distribution shifts, confounding variables, and dynamic environments. Causal reinforcement learning (CRL), leveraging the foundational principles of causal inference, offers promising solutions to these challenges by explicitly modeling cause-and-effect relationships. In this survey, we systematically review recent advancements at the intersection of causal inference and RL. We categorize existing approaches into causal representation learning, counterfactual policy optimization, offline causal RL, causal transfer learning, and causal explainability. Through this structured analysis, we identify prevailing challenges, highlight empirical successes in practical applications, and discuss open problems. Finally, we provide future research directions, underscoring the potential of CRL for developing robust, generalizable, and interpretable artificial intelligence systems.