Gibson Env: Real-World Perception for Embodied Agents

Developing visual perception models for active agents and sensorimotor control in the physical world are cumbersome as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a questi...

Full description

Saved in:
Bibliographic Details
Published in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition pp. 9068 - 9079
Main Authors Xia, Fei, Zamir, Amir R., He, Zhiyang, Sax, Alexander, Malik, Jitendra, Savarese, Silvio
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.06.2018
Subjects
Cameras
Neural networks
Physics
Rendering (computer graphics)
Robot sensing systems
Three-dimensional displays
Visualization
Online AccessGet full text

Cover

More Information
Summary:Developing visual perception models for active agents and sensorimotor control in the physical world are cumbersome as existing algorithms are too slow to efficiently learn in real-time and robots are fragile and costly. This has given rise to learning-in-simulation which consequently casts a question on whether the results transfer to real-world. In this paper, we investigate developing real-world perception for active agents, propose Gibson Environment for this purpose, and showcase a set of perceptual tasks learned therein. Gibson is based upon virtualizing real spaces, rather than artificially designed ones, and currently includes over 1400 floor spaces from 572 full buildings. The main characteristics of Gibson are: I. being from the real-world and reflecting its semantic complexity, II. having an internal synthesis mechanism "Goggles" enabling deploying the trained models in real-world without needing domain adaptation, III. embodiment of agents and making them subject to constraints of physics and space.
ISSN:1063-6919
DOI:10.1109/CVPR.2018.00945