A New Deep Q-Network Design for QoS Multicast Routing in Cognitive Radio MANETs

• Published in: IEEE access Vol. 9; p. 1
• Main Authors: Tran, Thong-Nhat, Nguyen, Toan-Van, Shim, Kyusung, Da Costa, Daniel Benevides, An, Beongku
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.01.2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Cognitive mobile ad hoc networks; Cognitive radio; Control stability; Costs; cross-layer; deep Q-network; Delay; Delays; Game theory; Mobile ad hoc networks; Mobile computing; Multicasting; Network design; Optimization; QoS multicast routing; Quality of service; Quality of service architectures; Queuing; Routing; Routing (telecommunications); Routing protocols; Shortest-path problems; Training; Trees
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2021.3126844

In this paper, we propose a new deep Q-network (DQN) design for quality-of-service (QoS) multicast routing (DQMR) protocol to establish efficient QoS multicast (EQM) trees in cognitive radio mobile ad hoc networks (CR-MANETs). An EQM tree is a shortest-path multicast tree with minimum end-to-end (E2E) cost (a combination of queuing size ratio and link stability) subject to QoS constraints such as queuing size ratio, link stability, number of hops, number of time slots and avoiding the licensed channel of primary users. Particularly, we propose a NP-complete optimization problem such that its feasible solution is an EQM tree. To address this problem, we design a new DQN model and a new game-based model to form EQM tree in real time by offline training instead of online training like previous papers. Moreover, the DQMR protocol is also guaranteed to have high stability, low routing delay, low control overhead, and high packet delivery ratio (PDR). Furthermore, one more new contribution of the paper is that exact closed-form expressions for the E2E queuing delay of a multicast routing tree are also derived assuming random waypoint mobility and the reference point group mobility models to compare with simulation results of routing delay. Simulation results show that the DQMR protocol outperforms multicast ad hoc on-demand distance vector routing protocol in terms of routing delay, control overhead, and PDR.