QoE-Driven Content-Centric Caching With Deep Reinforcement Learning in Edge-Enabled IoT

• Published in: IEEE computational intelligence magazine Vol. 14; no. 4; pp. 12 - 20
• Main Authors: He, Xiaoming, Wang, Kun, Xu, Wenyao
• Format: Magazine Article
• Language: English
• Published: Washington IEEE 01.11.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Architecture; Big Data; Bio-inspired computing; Biological system modeling; Caching; Computer simulation; Experts; Internet of Things; Knowledge management; Knowledge transfer; Machine learning; Reinforcement learning; Robot arms; Structural hierarchy; Task analysis
• ISSN: 1556-603X; 1556-6048; 1556-6048; 1556-603X
• DOI: 10.1109/MCI.2019.2937608

When humans learn several skills to solve multiple tasks, they exhibit an extraordinary capacity to transfer knowledge between them. The authors present here the last enhanced version of a bioinspired reinforcement-learning (RL) modular architecture able to perform skill-to-skill knowledge transfer and called transfer expert RL (TERL) model. TERL architecture is based on a RL actor-critic model where both actor and critic have a hierarchical structure, inspired by the mixture-of-experts model, formed by a gating network that selects experts specializing in learning the policies or value functions of different tasks. A key feature of TERL is the capacity of its gating networks to accumulate, in parallel, evidence on the capacity of experts to solve the new tasks so as to increase the responsibility for action of the best ones. A second key feature is the use of two different responsibility signals for the experts' functioning and learning: this allows the training of multiple experts for each task so that some of them can be later recruited to solve new tasks and avoid catastrophic interference. The utility of TERL mechanisms is shown with tests involving two simulated dynamic robot arms engaged in solving reaching tasks, in particular a planar 2-DoF arm, and a 3-D 4-DoF arm.