Hybrid Deep Reinforcement Learning for Enhancing Localization and Communication Efficiency in RIS-Aided Cooperative ISAC Systems

• Published in: IEEE internet of things journal Vol. 11; no. 18; pp. 29494 - 29510
• Main Authors: Saikia, Prajwalita, Singh, Keshav, Huang, Wan-Jen, Duong, Trung Q.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 15.09.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Array signal processing; Base stations; Beamforming; Communication; Deep learning; Hybrid deep reinforcement learning (HDRL); integrated sensing and communication (ISAC); Internet of Things; Localization; Location awareness; Machine learning; Markov processes; Performance evaluation; Phase shift; Position errors; Position measurement; Position sensing; Radar; Radio equipment; Reconfigurable intelligent surfaces; Resource efficiency; squared position error bound (SPEB); Wireless communication
• DOI: 10.1109/JIOT.2024.3411158
• ISSN: 2327-4662

In this article, we propose a novel framework that combines simultaneous localization and communication (SLAC) using a reconfigurable intelligent surface (RIS) aided integrated sensing and communication (ISAC) systems. Our primary focus is on enhancing resource efficiency in such systems. We introduce Cloud Radio Access Networks (C-RAN) that facilitate collaboration between multiple base stations (BSs), enhancing cooperation benefits for both communication and sensing capabilities. To evaluate localization performance, we formulate an optimization problem to minimize the squared position error bound (SPEB) that reflects the system functional performance by optimizing the transmit beamformer, phase shift and subcarrier assignment under certain constraints. Moreover, in order to adjust the phase shift of the RIS, we propose a RIS-aided cooperative ISAC SLAC protocol. This approach utilizes the measurements collected to refine the location and velocity estimates of the agent, as well as to reconstruct the environmental map with enhanced accuracy. However, the high dimensionality of the decision space makes the problem computationally intensive and challenging to navigate using gradient-based or exhaustive search methods. To efficiently tackle these issues, we construct a framework based on Markov decision processes (MDPs) and address it by introducing a novel algorithm called hybrid deep reinforcement learning (HDRL) algorithm. We validate our proposed algorithm through various simulations, demonstrating its effectiveness in improving system performance by comparing with the baseline schemes.