Gesture-based piloting of an aerial robot using monocular vision

• Published in: 2017 IEEE International Conference on Robotics and Automation (ICRA) pp. 5913 - 5920
• Main Authors: Ting Sun, Shengyi Nie, Dit-Yan Yeung, Shaojie Shen
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.05.2017
• Subjects: Algorithm design and analysis; Cameras; Gesture recognition; Hardware; Image color analysis; Robot vision systems; Unmanned aerial vehicles
• DOI: 10.1109/ICRA.2017.7989696

Aerial robots are becoming popular among general public, and with the development of artificial intelligence (AI), there is a trend to equip aerial robots with a natural user interface (NUI). Hand/arm gestures are an intuitive way to communicate for humans, and various research works have focused on controlling an aerial robot with natural gestures. However, the techniques in this area are still far from mature. Many issues in this area have been poorly addressed, such as the principles of choosing gestures from the design point of view, hardware requirements from an economic point of view, considerations of data availability, and algorithm complexity from a practical perspective. Our work focuses on building an economical monocular system particularly designed for gesture-based piloting of an aerial robot. Natural arm gestures are mapped to rich target directions and convenient fine adjustment is achieved. Practical piloting scenarios, hardware cost and algorithm applicability are jointly considered in our system design. The entire system is successfully implemented in an aerial robot and various properties of the system are tested.