Accurate manipulation of anthropomorphic robotic arm based on 3D human eye gazing with posture error compensation

• Published in: Science China. Technological sciences Vol. 68; no. 2
• Main Authors: Li, Quanlin, Wang, Xianzong, Ma, Jiaji, Huang, Qianjun, Jiang, Qizhi, Chen, Wenbin, Xiong, Caihua
• Format: Journal Article
• Language: English
• Published: Beijing Science China Press 01.02.2025; Springer Nature B.V
• Subjects: Anthropomorphism; Degrees of freedom; Engineering; Error compensation; Eye movements; Head movement; Human engineering; Light refraction; Man-machine interfaces; Optimization; Pupils; Real time; Research Paper; Robot arms; Robotics; Robots; anthropomorphic robotic arm; 3D gaze estimation; corneal refraction; head-mounted gaze tracker; differential joint; posture error compensation
• ISSN: 1674-7321; 1869-1900
• DOI: 10.1007/s11431-024-2861-8

As humans and robots work closer together than ever, anthropomorphic robotic arms with intuitive human-robot interaction interfaces have drawn massive attention to improving the quality of robot-assisted manipulation. In pursuit of this, we designed a dedicated 7-degrees-of-freedom (DoF) anthropomorphic robotic arm having three compact differential joints and a head-mounted gaze tracker enabling head-pose-tracked 3D gaze estimation. Moreover, two key challenges were addressed to achieve accurate robot-assisted manipulation of the object indicated by the direction of human gaze. First, a novel predictive pupil feature was proposed for 3D gaze estimation. Differing from most existing features subjected to the common paraxial approximation assumption, the proposed novel predictive pupil feature considered the light refraction at two corneal surfaces with a more realistic eye model, significantly improving the 3D gaze estimation accuracy when the eyeball rotates at large angles. Second, a novel optimization-based approach was developed to efficiently compensate for the posture errors of the designed 7-DoF anthropomorphic robotic arm for accurate manipulation. Compared with the existing Jacobian-based or optimization-based approaches with nominal joint values as iteration initial, the proposed approach computed the optimal iteration initial and realized faster convergence for real-time posture error compensation. With the posture error compensation in real time and 3D gaze estimated accurately, the human can command accurate robot-assisted manipulation using his eyes intuitively. The proposed system was successfully tested on five healthy subjects.