Visual Servo Based on YOLOX-Tiny and Low-Cost Robotic Manipulator with Joints Angle Estimation

• Published in: 2024 4th Asia-Pacific Conference on Communications Technology and Computer Science (ACCTCS) pp. 431 - 436
• Main Authors: Liang, Yida, Wang, Xiuping
• Format: Conference Proceeding
• Language: English
• Published: IEEE 24.02.2024
• Subjects: Automation; Estimation; IBVS; Inverse solution of robotic manipulator; Joint angle estimation; Manipulators; SIFT; Target recognition; Target tracking; Visual servo system; Visual systems; Visualization; YOLOX-Tiny
• DOI: 10.1109/ACCTCS61748.2024.00081

The robotic manipulator is being used in an increasingly wide range of fields requiring automation, especially to solve repetitive tasks. The visual servo system with visual feedback can control the robotic manipulator for object tracking, grabbing, and other actions. The visual servo system is mainly composed of a vision and robotic manipulator system. The vision system involves machine learning, deep learning models, etc. The robotic manipulator system involves the inverse kinematics of the robotic manipulator, joint angle estimation, etc. This paper uses the YOLOX-Tiny model in the visual servo system, and a low-cost steering robot arm scheme is adopted. YOLOX-Tiny and SIFT algorithms deployed in Jetson Nano can still achieve the visual system's sufficient frame rate recognition of target objects and feature points. For the robotic manipulator system, if the joint angle of the robotic manipulator composed of low-cost steering engines cannot be obtained, this paper uses the PWM duty cycle output by the controller to estimate it approximately. In the experiment, the two can better control the robotic manipulator, realize the tracking of the target object, and reflect the high control robustness.