Spectrum Access In Cognitive Radio Using a Two-Stage Reinforcement Learning Approach

• Published in: IEEE journal of selected topics in signal processing Vol. 12; no. 1; pp. 20 - 34
• Main Authors: Raj, Vishnu, Dias, Irene, Tholeti, Thulasi, Kalyani, Sheetal
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2018; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Bayes methods; Bayesian analysis; Bayesian learning; Cognitive radio; Computer simulation; Data transmission; Detection; Interference; Learning (artificial intelligence); Machine learning; multi-armed bandit; Multi-armed bandit problems; opportunistic spectrum sensing; reinforcement learning; Sensors; Signal processing algorithms; Throughput
• ISSN: 1932-4553; 1941-0484
• DOI: 10.1109/JSTSP.2018.2798920

With the advent of the fifth generation of wireless standards and an increasing demand for higher throughput, methods to improve spectral efficiency of wireless systems have become very important. In the context of cognitive radio, a substantial increase in throughput is possible if the secondary user can make smart decisions regarding which channel to sense and when or how often to sense. Here, we propose an algorithm to not only select a channel for data transmission, but also to predict how long the channel will remain unoccupied so that the time spent on channel sensing can be minimized. Our algorithm learns in two stages - a reinforcement learning approach for channel selection and a Bayesian approach to determine the duration for which sensing can be skipped. Comparisons with other methods are provided through extensive simulations. We show that the number of sensing operations is minimized with negligible increase in primary user interference; this implies that less energy is spent by the secondary user in sensing, and also higher throughput is achieved by saving the time spent on sensing.