Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments

TL;DR

This paper presents a deep reinforcement learning approach for adaptive radar control in congested spectral environments, improving detection and coexistence with communication systems.

Contribution

It introduces a Deep Q-Learning based method, including Double Deep Recurrent Q-Networks, for optimizing radar waveform parameters in real-time coexistence scenarios.

Findings

Enhanced SINR and bandwidth utilization over traditional methods

Demonstrated stability and performance of DDRQN in simulations

Validated approach on a software defined radar prototype

Abstract

In this paper, dynamic non-cooperative coexistence between a cognitive pulsed radar and a nearby communications system is addressed by applying nonlinear value function approximation via deep reinforcement learning (Deep RL) to develop a policy for optimal radar performance. The radar learns to vary the bandwidth and center frequency of its linear frequency modulated (LFM) waveforms to mitigate mutual interference with other systems and improve target detection performance while also maintaining sufficient utilization of the available frequency bands required for a fine range resolution. We demonstrate that our approach, based on the Deep Q-Learning (DQL) algorithm, enhances important radar metrics, including SINR and bandwidth utilization, more effectively than policy iteration or sense-and-avoid (SAA) approaches in a variety of realistic coexistence environments. We also extend the DQL-based approach to incorporate Double Q-learning and a recurrent neural network to form a Double Deep Recurrent Q-Network (DDRQN). We demonstrate the DDRQN results in favorable performance and stability compared to DQL and policy iteration. Finally, we demonstrate the practicality of our proposed approach through a discussion of experiments performed on a software defined radar (SDRadar) prototype system. Our experimental results indicate that the proposed Deep RL approach significantly improves radar detection performance in congested spectral environments when compared to policy iteration and SAA.