Real-time human hand following system based on multiple vision sensors

This paper designs a real-time human hand following system based on multi-vision sensors. Through the KinectV2 camera, human hand behavior can be observed; the pose information of the human hand joint can also be obtained by using OpenNI2 and NITE2 interface; calculating the angle information of eac...

Full description

Saved in:
Bibliographic Details
Published in2019 3rd International Conference on Electronic Information Technology and Computer Engineering (EITCE) pp. 1641 - 1645
Main Authors Yiqiao, Zhou, Yulin, Xu, Hong, Xu
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.10.2019
Subjects
Fingers
Handheld computers
Kalman filters
KinectV2
LeapMotion
Manipulators
Robot Operation System(ROS)
Skeleton
skeleton following
Online AccessGet full text

Cover

More Information
Summary:This paper designs a real-time human hand following system based on multi-vision sensors. Through the KinectV2 camera, human hand behavior can be observed; the pose information of the human hand joint can also be obtained by using OpenNI2 and NITE2 interface; calculating the angle information of each joint of the six-degree-of-freedom robot can be achieved by stereo geometric correlation calculation. Through LeapMotion, accurate human joint information is obtained, then calculating the finger distance to control manipulator's tension, which is smoothed by Kalman filtering. Control, simulation and visualization for real-time follow-up of the robotic arm and the human hand are achieved under the Robot Operating System(ROS). The final experimental results are verified by Moveit's simulation environment and Kinova's Jaco arm.
DOI:10.1109/EITCE47263.2019.9095182