Algorithmic Contract Design with Reinforcement Learning Agents
We introduce a novel problem setting for algorithmic contract design, named the principal-MARL contract design problem. This setting extends traditional contract design to account for dynamic and stochastic environments using Markov Games and Multi-Agent Reinforcement Learning. To tackle this proble...
Saved in:
Main Authors | , , , , |
---|---|
Format | Journal Article |
Language | English |
Published |
18.08.2024
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | We introduce a novel problem setting for algorithmic contract design, named
the principal-MARL contract design problem. This setting extends traditional
contract design to account for dynamic and stochastic environments using Markov
Games and Multi-Agent Reinforcement Learning. To tackle this problem, we
propose a Multi-Objective Bayesian Optimization (MOBO) framework named
Constrained Pareto Maximum Entropy Search (cPMES). Our approach integrates MOBO
and MARL to explore the highly constrained contract design space, identifying
promising incentive and recruitment decisions. cPMES transforms the
principal-MARL contract design problem into an unconstrained multi-objective
problem, leveraging the probability of feasibility as part of the objectives
and ensuring promising designs predicted on the feasibility border are included
in the Pareto front. By focusing the entropy prediction on designs within the
Pareto set, cPMES mitigates the risk of the search strategy being overwhelmed
by entropy from constraints. We demonstrate the effectiveness of cPMES through
extensive benchmark studies in synthetic and simulated environments, showing
its ability to find feasible contract designs that maximize the principal's
objectives. Additionally, we provide theoretical support with a sub-linear
regret bound concerning the number of iterations. |
---|---|
DOI: | 10.48550/arxiv.2408.09686 |