An Exploration of Embodied Visual Exploration

Embodied computer vision considers perception for robots in novel, unstructured environments. Of particular importance is the embodied visual exploration problem: how might a robot equipped with a camera scope out a new environment? Despite the progress thus far, many basic questions pertinent to th...

Full description

Saved in:
Bibliographic Details
Published inInternational journal of computer vision Vol. 129; no. 5; pp. 1616 - 1649
Main Authors Ramakrishnan, Santhosh K., Jayaraman, Dinesh, Grauman, Kristen
Format Journal Article
LanguageEnglish
Published New York Springer US 01.05.2021
Springer
Springer Nature B.V
Subjects
Algorithms
Artificial Intelligence
Computer Imaging
Computer Science
Computer vision
Exploration
Image Processing and Computer Vision
Machine vision
Pattern Recognition
Pattern Recognition and Graphics
Performance measurement
Questions
Robots
Special Issue on Performance Evaluation in Computer Vision
Taxonomy
Vision
Visual navigation
Visual exploration
Learning for navigation
Online AccessGet full text

Cover

More Information
Summary:Embodied computer vision considers perception for robots in novel, unstructured environments. Of particular importance is the embodied visual exploration problem: how might a robot equipped with a camera scope out a new environment? Despite the progress thus far, many basic questions pertinent to this problem remain unanswered: (i) What does it mean for an agent to explore its environment well? (ii) Which methods work well, and under which assumptions and environmental settings? (iii) Where do current approaches fall short, and where might future work seek to improve? Seeking answers to these questions, we first present a taxonomy for existing visual exploration algorithms and create a standard framework for benchmarking them. We then perform a thorough empirical study of the four state-of-the-art paradigms using the proposed framework with two photorealistic simulated 3D environments, a state-of-the-art exploration architecture, and diverse evaluation metrics. Our experimental results offer insights and suggest new performance metrics and baselines for future work in visual exploration. Code, models and data are publicly available.
ISSN:0920-5691
1573-1405
DOI:10.1007/s11263-021-01437-z