Digital Twin-assisted belief-state reinforcement learning for latency-robust ISAC in 6G networks

TL;DR

This paper introduces a Digital Twin-assisted reinforcement learning framework to enhance latency-robust integrated sensing and communication in 6G networks, effectively managing telemetry delays for stable and efficient operation.

Contribution

It develops a novel belief-state reinforcement learning approach utilizing a Digital Twin to reconstruct synchronized states from delayed telemetry, improving ISAC performance under latency.

Findings

Achieves 12% median throughput improvement at 50 ms latency.

Reduces sensing error by 7% compared to baseline.

Maintains 88% of zero-latency throughput at 100 ms delay.

Abstract

Integrated Sensing and Communication (ISAC) enables joint data transmission and environmental perception for sixth-generation (6G) networks, but centralized and virtualized RAN control loops introduce telemetry latency that yields stale observations and unstable control. This paper proposes a Digital Twin-assisted belief-state reinforcement learning framework for latency-robust ISAC. A Digital Twin (DT) reconstructs a synchronized belief state from delayed telemetry using an Extended Kalman Filter, and a Proximal Policy Optimization agent performs joint beamforming and power allocation for communication and sensing. Closed-loop simulations with telemetry delays up to 100 ms demonstrate consistent performance gains over latency-unaware deep reinforcement learning (DRL) and heuristic baselines. At 50 ms latency, the proposed method improves median throughput by 12% and reduces sensing error by 7% relative to a DT-only controller, while achieving an order-of-magnitude reduction in reliability violations. Even at 100 ms latency, the proposed approach retains approximately 88% of its zero-latency throughput. These results show that Digital Twin-assisted belief-state control enables stable and efficient ISAC operation under realistic telemetry delays in 6G networks.