Continual Deep Reinforcement Learning to Prevent Catastrophic Forgetting in Jamming Mitigation

TL;DR

This paper introduces a continual deep reinforcement learning approach based on PackNet to prevent catastrophic forgetting in anti-jamming systems, enhancing adaptability and robustness in dynamic wireless environments.

Contribution

It presents a novel continual DRL method that retains knowledge of past jammer patterns while learning new ones, improving anti-jamming performance in changing environments.

Findings

Significant reduction in catastrophic forgetting using the proposed method

Enhanced anti-jamming performance over standard DRL approaches

Improved adaptability and robustness in dynamic RF environments

Abstract

Deep Reinforcement Learning (DRL) has been highly effective in learning from and adapting to RF environments and thus detecting and mitigating jamming effects to facilitate reliable wireless communications. However, traditional DRL methods are susceptible to catastrophic forgetting (namely forgetting old tasks when learning new ones), especially in dynamic wireless environments where jammer patterns change over time. This paper considers an anti-jamming system and addresses the challenge of catastrophic forgetting in DRL applied to jammer detection and mitigation. First, we demonstrate the impact of catastrophic forgetting in DRL when applied to jammer detection and mitigation tasks, where the network forgets previously learned jammer patterns while adapting to new ones. This catastrophic interference undermines the effectiveness of the system, particularly in scenarios where the environment is non-stationary. We present a method that enables the network to retain knowledge of old jammer patterns while learning to handle new ones. Our approach substantially reduces catastrophic forgetting, allowing the anti-jamming system to learn new tasks without compromising its ability to perform previously learned tasks effectively. Furthermore, we introduce a systematic methodology for sequentially learning tasks in the anti-jamming framework. By leveraging continual DRL techniques based on PackNet, we achieve superior anti-jamming performance compared to standard DRL methods. Our proposed approach not only addresses catastrophic forgetting but also enhances the adaptability and robustness of the system in dynamic jamming environments. We demonstrate the efficacy of our method in preserving knowledge of past jammer patterns, learning new tasks efficiently, and achieving superior anti-jamming performance compared to traditional DRL approaches.