Causality-Driven Reinforcement Learning for Joint Communication and Sensing

TL;DR

This paper introduces a causally-aware reinforcement learning framework for joint communication and sensing in 6G mMIMO systems, improving beamforming efficiency by discovering causal relationships during training.

Contribution

It proposes a novel causally-aware RL approach with state-dependent action selection for mMIMO JCAS, enhancing learning efficiency and beamforming performance.

Findings

Causally-aware RL outperforms baseline methods in beamforming gain.

The framework reduces training overhead in mMIMO JCAS systems.

Causal discovery improves environmental sensing accuracy.

Abstract

The next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems realize this integration for 6G applications such as autonomous driving, as it requires accurate environmental sensing and time-critical communication with neighboring vehicles. Reinforcement Learning (RL) is used for mMIMO antenna beamforming in the existing literature. However, the huge search space for actions associated with antenna beamforming causes the learning process for the RL agent to be inefficient due to high beam training overhead. The learning process does not consider the causal relationship between action space and the reward, and gives all actions equal importance. In this work, we explore a causally-aware RL agent which can intervene and discover causal relationships for mMIMO-based JCAS environments, during the training phase. We use a state dependent action dimension selection strategy to realize causal discovery for RL-based JCAS. Evaluation of the causally-aware RL framework in different JCAS scenarios shows the benefit of our proposed framework over baseline methods in terms of the beamforming gain.