DRDC: Deep reinforcement learning based duty cycle for energy harvesting body sensor node

• Published in: Energy reports Vol. 9; pp. 1707 - 1719
• Main Authors: Mohammadi, Razieh, Shirmohammadi, Zahra
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.12.2023; Elsevier
• Subjects: Body sensor node; Deep reinforcement learning; Duty cycle; Energy efficiency; Energy harvesting; Body sensor node; Deep reinforcement learning; Energy efficiency; Energy harvesting; Duty cycle
• ISSN: 2352-4847; 2352-4847
• DOI: 10.1016/j.egyr.2022.12.138

Nowadays, wireless body area networks are the center of attention for patients’ health data monitoring. In these networks, the sensor nodes deal with the limited energy of batteries to provide sustainable services. Unlimited energy supply by energy harvesters in these nodes as complementary to batteries makes perform in a stable state. However, the harvested energy has an irregular and unpredictable rate. When the data sensed by the sensor is stable, the continuous activity of the energy-harvesting body node (EH-BN) is necessary. Also, it can fully discharge the EH-BN energy if the harvestable energy rate is low. Therefore, it is required that EH-BN to be in sleep and wake modes periodically to adjust the duty cycle for EH-BN. Reinforcement learning algorithms perform nicely in determining the duty cycle under uncertain conditions. Previous RL-based methods for determining the BN’s duty cycle have two fundamental problems: (1) BN suffers from the emergency packet loss or unnecessary frequent sleeping and waking, and (2) discretization of the problem space does not ensure the determination of the optimal duty cycle. This paper proposes a new method for determining the EH-BN’s duty cycle based on deep reinforcement learning (DRDC). The novelties of DRDC are as follows: (1) considers the change rate of data sensed by BN in addition to its energy to avoid the emergency packet loss and unnecessary frequent sleep/wake, (2) uses Deep Q-Network (DQN) with light neural network for accurately determining BN’s duty cycle, (3) applies a three-layer communication architectural model when there are extreme limitations in BN resources to preserve the memory constraints and computational power of EH-BN. In this architectural model, the algorithm is executed on a local server, and only the trained policy is transmitted to EH-BN. (4) designs a reward function to realize the suitable performance of the DQN algorithm. This function simultaneously depends on the EH-BN energy, change rate of the sensed data, and sleep time and can balance these parameters. Results of simulations suggest that the proposed method decreases the EH-BN duty cycle by about 28% and the data overhead by more than 50% on average relative to similar studies.