A Novel Nested Q-Learning Method to Tackle Time-Constrained Competitive Influence Maximization

• Published in: IEEE access Vol. 7; pp. 6337 - 6352
• Main Authors: Ali, Khurshed, Wang, Chih-Yu, Chen, Yi-Shin
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Budgets; Companies; Competition; competitive influence maximization; Concerts; Constraints; Greedy algorithms; influence maximization; Integrated circuit modeling; Machine learning; Maximization; Optimization; reinforcement learning; Scalability; Social network analysis; Social network services; Time factors
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2018.2888895

Time plays a critical role in competitive influence maximization. Companies aim to promote their products before certain events, such as Christmas Eve or music concerts, to gain more benefit under competitions from other companies. Besides, these companies have a limited budget to spend on these product promotions. Therefore, in this paper, we examine a time-constrained competitive influence maximization where the parties wish to maximize their profits before the respective deadlines. Besides, the parties need to determine how to select the seed nodes and when to initiate information propagation in the network, such that the decision results in the optimal reward given the time and the budget constraint. To this end, we propose a novel reinforcement learning-based framework named seed-combination and seed-selection that is built on a nested Q-learning (NSQ) algorithm. This way, we can derive the optimal in both budget allocation and node selection that results in the maximum profit. In evaluating the proposed model, we consider the scenarios when the competitors' strategy is known, unknown, and not available for training. The results show that the proposed NSQ algorithm could improve the rewards by up to 50% compared with the state-of-the-art algorithm, STORM-Q.