Introspective perception: Learning to predict failures in vision systems

• Published in: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) pp. 1743 - 1750
• Main Authors: Daftry, Shreyansh, Zeng, Sam, Bagnell, J. Andrew, Hebert, Martial
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2016
• Subjects: Context; Prediction algorithms; Reliability; Robot sensing systems; Support vector machines; Training
• ISSN: 2153-0866
• DOI: 10.1109/IROS.2016.7759279

As robots aspire for long-term autonomous operations in complex dynamic environments, the ability to reliably take mission-critical decisions in ambiguous situations becomes critical. This motivates the need to build systems that have situational awareness to assess how qualified they are at that moment to make a decision. We call this self-evaluating capability as introspection. In this paper, we take a small step in this direction and propose a generic framework for introspective behavior in perception systems. Our goal is to learn a model to reliably predict failures in a given system, with respect to a task, directly from input sensor data. We present this in the context of vision-based autonomous MAV flight in outdoor natural environments, and show that it effectively handles uncertain situations.