Learning for Multi-robot Cooperation in Partially Observable Stochastic Environments with Macro-actions

This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Liu, Miao, Sivakumar, Kavinayan, Shayegan Omidshafiei, Amato, Christopher, How, Jonathan P
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 18.08.2017
Subjects
Algorithms
Computer simulation
Cooperation
Decision making
Domains
Hardware
Initial conditions
Iterative methods
Machine learning
Markov processes
Multiple robots
Optimization
Robots
Search and rescue missions
Online AccessGet full text

Cover

More Information
Summary:This paper presents a data-driven approach for multi-robot coordination in partially-observable domains based on Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) and macro-actions (MAs). Dec-POMDPs provide a general framework for cooperative sequential decision making under uncertainty and MAs allow temporally extended and asynchronous action execution. To date, most methods assume the underlying Dec-POMDP model is known a priori or a full simulator is available during planning time. Previous methods which aim to address these issues suffer from local optimality and sensitivity to initial conditions. Additionally, few hardware demonstrations involving a large team of heterogeneous robots and with long planning horizons exist. This work addresses these gaps by proposing an iterative sampling based Expectation-Maximization algorithm (iSEM) to learn polices using only trajectory data containing observations, MAs, and rewards. Our experiments show the algorithm is able to achieve better solution quality than the state-of-the-art learning-based methods. We implement two variants of multi-robot Search and Rescue (SAR) domains (with and without obstacles) on hardware to demonstrate the learned policies can effectively control a team of distributed robots to cooperate in a partially observable stochastic environment.
ISSN:2331-8422