Hierarchical Deep Reinforcement Learning for Robust Access in Cognitive IoT Networks under Smart Jamming Attacks

TL;DR

This paper introduces a hierarchical deep reinforcement learning framework to enable a cognitive IoT device to make complex, multi-level decisions for spectrum access and energy harvesting under smart jamming attacks, improving robustness and performance.

Contribution

The paper proposes a novel hierarchical DDPG-based method for joint decision-making in cognitive IoT networks under adversarial jamming, addressing hybrid action spaces and multi-level control.

Findings

H-DDPG outperforms flat RL baselines in simulations.

The hierarchical approach effectively manages hybrid discrete-continuous actions.

The method enhances robustness against smart jamming attacks.

Abstract

In this paper, we address the challenge of dynamic spectrum access in a cognitive Internet of Things (CIoT) network where a secondary user (SU) operates under both energy constraints and adversarial interference from a smart jammer. The SU coexists with primary users (PUs) and must ensure that its transmissions do not exceed a predefined interference threshold on licensed channels. At each time slot, the SU must jointly determine whether to transmit or harvest energy, which channel to access, and the appropriate transmit power while satisfying energy and interference constraints. Meanwhile, a smart jammer actively selects a channel to disrupt, aiming to degrade the SU's communication performance. This setting presents a significant challenge due to its multi-level decision structure and hybrid action space, which combines both discrete and continuous decisions. To tackle this, we propose a novel Hierarchical Deep Deterministic Policy Gradient (H-DDPG) framework that decomposes the decision-making process into three levels: the high-level policy determines the mode (transmit or harvest), the mid-level policy selects the channel, and the low-level actor outputs a continuous power level. Concurrently, the jammer is modeled as a reinforcement learning agent that learns an adaptive channel jamming strategy using a discrete variant of DDPG. Simulation results show that our H-DDPG approach outperforms conventional flat reinforcement learning baselines.