Deep Reinforcement Learning Control for Radar Detection and Tracking in Congested Spectral Environments

• Published in: IEEE transactions on cognitive communications and networking Vol. 6; no. 4; pp. 1335 - 1349
• Main Authors: Thornton, Charles E., Kozy, Mark A., Buehrer, R. Michael, Martone, Anthony F., Sherbondy, Kelly D.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.12.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Cognitive radar; Deep learning; Deep reinforcement learning; Frequencies; Interference; Iterative methods; Machine learning; Markov decision process; Optimization; Performance measurement; Radar detection; Radar tracking; Recurrent neural networks; spectrum sharing; Target detection; Waveforms
• ISSN: 2332-7731; 2332-7731
• DOI: 10.1109/TCCN.2020.3019605

This work addresses dynamic non-cooperative coexistence between a cognitive pulsed radar and nearby communications systems by applying nonlinear value function approximation via deep reinforcement learning (Deep RL) to develop a policy for optimal radar performance. The radar learns to vary the bandwidth and center frequency of its linear frequency modulated (LFM) waveforms to mitigate interference with other systems for improved target detection performance while also sufficiently utilizing available frequency bands to achieve a fine range resolution. We demonstrate that this approach, based on the Deep <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-Learning (DQL) algorithm, enhances several radar performance metrics more effectively than policy iteration or sense-and-avoid (SAA) approaches in several realistic coexistence environments. The DQL-based approach is also extended to incorporate Double <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-learning and a recurrent neural network to form a Double Deep Recurrent <inline-formula> <tex-math notation="LaTeX">{Q} </tex-math></inline-formula>-Network (DDRQN), which yields favorable performance and stability compared to DQL and policy iteration. The practicality of the proposed scheme is demonstrated through experiments performed on a software defined radar (SDRadar) prototype system. Experimental results indicate that the proposed Deep RL approach significantly improves radar detection performance in congested spectral environments compared to policy iteration and SAA.