ROBOVERINE: A human-inspired neural robotic process model of active visual search and scene grammar in naturalistic environments

• Published in: Proceedings of the ... IEEE/RSJ International Conference on Intelligent Robots and Systems pp. 11470 - 11477
• Main Authors: Grieben, Raul, Sehring, Stephan, Tekulve, Jan, Spencer, John P., Schoner, Gregor
• Format: Conference Proceeding
• Language: English
• Published: IEEE 14.10.2024
• Subjects: Analytical models; Cognitive robotics; Feature extraction; Grammar; Intelligent robots; Robot kinematics; Robot vision systems; Search problems; Switches; Visualization
• ISSN: 2153-0866
• DOI: 10.1109/IROS58592.2024.10801621

We present ROBOVERINE, a neural dynamic robotic active vision process model of selective visual attention and scene grammar in naturalistic environments. The model addresses significant challenges for cognitive robotic models of visual attention: combined bottom-up salience and top-down feature guidance, combined overt and covert attention, coordinate transformations, two forms of inhibition of return, finding objects outside of the camera frame, integrated space-and object-based analysis, minimally supervised few-shot continuous online learning for recognition and guidance templates, and autonomous switching between exploration and visual search. Furthermore, it incorporates a neural process account of scene grammar - prior knowledge about the relation between objects in the scene - to reduce the search space and increase search efficiency. The model also showcases the strength of bridging two frameworks: Deep Neural Networks for feature extractions and Dynamic Field Theory for cognitive operations.