DRL-Based Robust Multi-Timescale Anti-Jamming Approaches under State Uncertainty

TL;DR

This paper introduces robust deep reinforcement learning algorithms for anti-jamming in wireless systems, addressing state uncertainty and sensor errors to improve reliability under imperfect sensing conditions.

Contribution

It proposes two novel DRL-based schemes, PGD-DDQN and NQC-DDQN, that enhance robustness against sensing errors in multi-timescale anti-jamming scenarios.

Findings

Algorithms maintain performance with minor degradation under perturbations.

Proposed methods outperform baseline in imperfect sensing conditions.

Simulation validates robustness and practicality of the approaches.

Abstract

Owing to the openness of wireless channels, wireless communication systems are highly susceptible to malicious jamming. Most existing anti-jamming methods rely on the assumption of accurate sensing and optimize parameters on a single timescale. However, such methods overlook two practical issues: mismatched execution latencies across heterogeneous actions and measurement errors caused by sensor imperfections. Especially for deep reinforcement learning (DRL)-based methods, the inherent sensitivity of neural networks implies that even minor perturbations in the input can mislead the agent into choosing suboptimal actions, with potentially severe consequences. To ensure reliable wireless transmission, we establish a multi-timescale decision model that incorporates state uncertainty. Subsequently, we propose two robust schemes that sustain performance under bounded sensing errors. First, a Projected Gradient Descent-assisted Double Deep Q-Network (PGD-DDQN) algorithm is designed, which derives worst-case perturbations under a norm-bounded error model and applies PGD during training for robust optimization. Second, a Nonlinear Q-Compression DDQN (NQC-DDQN) algorithm introduces a nonlinear compression mechanism that adaptively contracts Q-value ranges to eliminate action aliasing. Simulation results indicate that, compared with the perfect-sensing baseline, the proposed algorithms show only minor degradation in anti-jamming performance while maintaining robustness under various perturbations, thereby validating their practicality in imperfect sensing conditions.