Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledge

TL;DR

This paper introduces a causal structure-aware reinforcement learning framework for healthcare robots, improving decision-making interpretability, robustness, and efficiency in complex patient interactions, validated through simulated cognitive care scenarios.

Contribution

The paper presents a novel CRL framework integrating causal discovery into policy optimization, enhancing interpretability and robustness in healthcare robot interactions.

Findings

CRL outperforms conventional RL in simulated cognitive care tasks.

CRL maintains more consistent and clinically-aligned patient states.

Performance remains robust across different hyperparameters.

Abstract

Reinforcement Learning (RL) faces significant challenges in adaptive healthcare interventions, such as dementia care, where data is scarce, decisions require interpretability, and underlying patient-state dynamic are complex and causal in nature. In this work, we present a novel framework called Causal structure-aware Reinforcement Learning (CRL) that explicitly integrates causal discovery and reasoning into policy optimization. This method enables an agent to learn and exploit a directed acyclic graph (DAG) that describes the causal dependencies between human behavioral states and robot actions, facilitating more efficient, interpretable, and robust decision-making. We validate our approach in a simulated robot-assisted cognitive care scenario, where the agent interacts with a virtual patient exhibiting dynamic emotional, cognitive, and engagement states. The experimental results show that CRL agents outperform conventional model-free RL baselines by achieving higher cumulative rewards, maintaining desirable patient states more consistently, and exhibiting interpretable, clinically-aligned behavior. We further demonstrate that CRL's performance advantage remains robust across different weighting strategies and hyperparameter settings. In addition, we demonstrate a lightweight LLM-based deployment: a fixed policy is embedded into a system prompt that maps inferred states to actions, producing consistent, supportive dialogue without LLM finetuning. Our work illustrates the promise of causal reinforcement learning for human-robot interaction applications, where interpretability, adaptiveness, and data efficiency are paramount.