RLBench: The Robot Learning Benchmark & Learning Environment

• Published in: IEEE robotics and automation letters Vol. 5; no. 2; pp. 3018 - 3025
• Main Authors: James, Stephen, Ma, Zicong, Arrojo, David Rovick, Davison, Andrew J.
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.04.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Benchmark testing; Benchmarks; Cameras; Computer vision; deep learning in robotics and automation; Learning (artificial intelligence); Learning from demonstration; Masks; perception for grasping and manipulation; performance evaluation and benchmarking; Robot learning; Robot vision systems; Robotics; Robots; Segmentation; Task analysis; Visual observation; Waypoints
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2020.2974707

We present a challenging new benchmark and learning-environment for robot learning: RLBench. The benchmark features 100 completely unique, hand-designed tasks, ranging in difficulty from simple target reaching and door opening to longer multi-stage tasks, such as opening an oven and placing a tray in it. We provide an array of both proprioceptive observations and visual observations, which include rgb, depth, and segmentation masks from an over-the-shoulder stereo camera and an eye-in-hand monocular camera. Uniquely, each task comes with an infinite supply of demos through the use of motion planners operating on a series of waypoints given during task creation time; enabling an exciting flurry of demonstration-based learning possibilities. RLBench has been designed with scalability in mind; new tasks, along with their motion-planned demos, can be easily created and then verified by a series of tools, allowing users to submit their own tasks to the RLBench task repository. This large-scale benchmark aims to accelerate progress in a number of vision-guided manipulation research areas, including: reinforcement learning, imitation learning, multi-task learning, geometric computer vision, and in particular, few-shot learning. With the benchmark's breadth of tasks and demonstrations, we propose the first large-scale few-shot challenge in robotics. We hope that the scale and diversity of RLBench offers unparalleled research opportunities in the robot learning community and beyond. Benchmarking code and videos can be found at https://sites.google.com/view/rlbench .