Hypergraph-Based Model for Modeling Multi-Agent Q-Learning Dynamics in Public Goods Games

Modeling the learning dynamic of multi-agent systems has long been a crucial issue for understanding the emergence of collective behavior. In public goods games, agents interact in multiple larger groups. While previous studies have primarily focused on infinite populations that only allow pairwise...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on network science and engineering Vol. 11; no. 6; pp. 6169 - 6179
Main Authors Shi, Juan, Liu, Chen, Liu, Jinzhuo
Format Journal Article
LanguageEnglish
Published Piscataway IEEE 01.11.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Biological system modeling
Distribution functions
Fokker-Planck equation
Game theory
Games
Graph theory
Graphs
Heuristic algorithms
hypergraph
Learning
Mathematical models
Modelling
multi-agent system
Multi-agent systems
Multiagent systems
Parameters
Q-learning
System dynamics
Topology
Online AccessGet full text

Cover

More Information
Summary:Modeling the learning dynamic of multi-agent systems has long been a crucial issue for understanding the emergence of collective behavior. In public goods games, agents interact in multiple larger groups. While previous studies have primarily focused on infinite populations that only allow pairwise interactions, we aim to investigate the learning dynamics of agents in a public goods game with higher-order interactions. With a novel use of hypergraphs for encoding higher-order interactions, we develop a formal model (a Fokker-Planck equation) to describe the temporal evolution of the distribution function of Q-values. Noting that early research focused on replicator models to predict system dynamics failed to accurately capture the impact of hyperdegree in hypergraphs, our model effectively maps its influence. Through experiments, we demonstrate that our theoretical findings are consistent with the agent-based simulation results. We demonstrated that as the number of groups an agent is involved in reaches a certain scale, the learning dynamics of the system evolve to resemble those of a well-mixed population. Furthermore, we demonstrate that our model offers insights into algorithmic parameters, such as the Boltzmann temperature, facilitating parameter tuning.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2327-4697
2334-329X
DOI:10.1109/TNSE.2024.3473941