A robot arm digital twin utilising reinforcement learning

• Published in: Computers & graphics Vol. 95; pp. 106 - 114
• Main Authors: Matulis, Marius, Harvey, Carlo
• Format: Journal Article
• Language: English
• Published: Oxford Elsevier Ltd 01.04.2021; Elsevier Science Ltd
• Subjects: Artificial intelligence; Digital twin; Digital twins; Domains; Industrial applications; Learning; Reinforcement learning; Robot arm; Robot arms; Robots; Three dimensional printing; Training; Robot arm; Digital twin; Artificial intelligence; Reinforcement learning
• ISSN: 0097-8493; 1873-7684
• DOI: 10.1016/j.cag.2021.01.011

•The manuscript investigates a digital twin of a robot arm typical of manufacturing processes.•This explores the virtual construction of the training and evaluation environments as well as the physical equivalency of this digital twin.•We train, using reinforcement learning, a robot arm to perform a task in virtual space and then map this learnt behaviour to a physical counterpart whilst illustrating linkages between the halves of the digital twin. [Display omitted] For many industry contexts, the implementation of Artificial Intelligence (AI) has contributed to what has become known as the fourth industrial revolution or “Industry 4.0” and creates an opportunity to deliver significant benefit to both businesses and their stakeholders. Robot arms are one of the most common devices utilised in manufacturing and industrial processes, used for a wide variety of automation tasks on, for example, a factory floor but the effective use of these devices requires AI to be appropriately trained. One approach to support AI training of these devices is the use of a “Digital Twin”. There are, however, a number of challenges that exist within this domain, in particular, success depends upon the ability to collect data of what are considered as observations within the environment and the application of one or many trained AI policies to the task that is to be completed. This project presents a case-study of creating and training a Robot Arm Digital Twin as an approach for AI training in a virtual space and applying this simulation learning within physical space. A virtual space, created using Unity (a contemporary Game Engine), incorporating a virtual robot arm was linked to a physical space, being a 3D printed replica of the virtual space and robot arm. These linked environments were applied to solve a task and provide training for an AI model. The contribution of this work is to provide guidance on training protocols for a digital twin together with details of the necessary architecture to support effective simulation in a virtual space through the use of Tensorflow and hyperparameter tuning. It provides an approach to addressing the mapping of learning in the virtual domain to the physical robot twin.