Space Debris Removal: Learning to Cooperate and the Price of Anarchy

• Published in: Frontiers in robotics and AI Vol. 5; p. 54
• Main Authors: Klima, Richard, Bloembergen, Daan, Savani, Rahul, Tuyls, Karl, Wittig, Alexander, Sapera, Andrei, Izzo, Dario
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 04.06.2018
• Subjects: active debris removal; markov decision process; price of anarchy; Robotics and AI; space debris; tragedy of the commons; price of anarchy; tragedy of the commons; space debris; markov decision process; active debris removal
• ISSN: 2296-9144; 2296-9144
• DOI: 10.3389/frobt.2018.00054

In this paper we study space debris removal from a game-theoretic perspective. In particular we focus on the question whether and how self-interested agents can cooperate in this dilemma, which resembles a tragedy of the commons scenario. We compare centralised and decentralised solutions and the corresponding price of anarchy, which measures the extent to which competition approximates cooperation. In addition we investigate whether agents can learn optimal strategies by reinforcement learning. To this end, we improve on an existing high fidelity orbital simulator, and use this simulator to obtain a computationally efficient surrogate model that can be used for our subsequent game-theoretic analysis. We study both single- and multi-agent approaches using stochastic (Markov) games and reinforcement learning. The main finding is that the cost of a decentralised, competitive solution can be significant, which should be taken into consideration when forming debris removal strategies.