Stateless Q-learning algorithm for service caching in resource constrained edge environment

• Published in: Journal of cloud computing : advances, systems and applications Vol. 12; no. 1; pp. 132 - 13
• Main Authors: Huang, Binbin, Ran, Ziqi, Yu, Dongjin, Xiang, Yuanyuan, Shi, Xiaoying, Li, Zhongjin, Xu, Zhengqian
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.12.2023; Springer Nature B.V; SpringerOpen
• Subjects: AI-empowered Resource Provision and Service Scheduling in Multi-Clouds; Algorithms; Caching; Collaboration; Collaboration cost; Computer Communication Networks; Computer Science; Computer System Implementation; Computer Systems Organization and Communication Networks; Cooperation; Edge environment; End users; Information Systems Applications (incl.Internet); Machine learning; Quality of service architectures; Servers; service caching; Service latency; Software Engineering/Programming and Operating Systems; Special Purpose and Application-Based Systems; Stateless Q-learning; Statelessness; Edge environment; Stateless Q-learning; Service latency; Collaboration cost; service caching
• ISSN: 2192-113X; 2192-113X
• DOI: 10.1186/s13677-023-00506-7

In resource constrained edge environment, multiple service providers can compete to rent the limited resources to cache their service instances on edge servers close to end users, thereby significantly reducing the service delay and improving quality of service (QoS). However, service providers renting the resources of different edge servers to deploy their service instances can incur different resource usage costs and service delay. To make full use of the limited resources of the edge servers to further reduce resource usage costs, multiple service providers on an edge server can form a coalition and share the limited resource of an edge server. In this paper, we investigate the service caching problem of multiple service providers in resource constrained edge environment, and propose an independent learners-based services caching scheme (ILSCS) which adopts a stateless Q-learning to learn an optimal service caching scheme. To verify the effectiveness of ILSCS scheme, we implement COALITION, RANDOM, MDU, and MCS four baseline algorithms, and compare the total collaboration cost and service latency of ILSCS scheme with these of these four baseline algorithms under different experimental parameter settings. The extensive experimental results show that the ILSCS scheme can achieve lower total collaboration cost and service latency.