Robotic teaching through presentation with visual servo

• Main Authors: WANG KAIMENG, KATO TETSURO
• Format: Patent
• Language: Chinese; English
• Published: 06.06.2023
• Subjects: CHAMBERS PROVIDED WITH MANIPULATION DEVICES; HAND TOOLS; MANIPULATORS; PERFORMING OPERATIONS; PORTABLE POWER-DRIVEN TOOLS; TRANSPORTING

A method for teaching and controlling a robot to perform operations based on a human presentation utilizing images from a camera. The method includes a presentation phase in which a camera detects a coarse trajectory of robot movement of a hand gripping and moving a workpiece to define the workpiece. Line features or other geometric features on the workpiece collected during the presentation phase are used for image-based visual servo (IBVS) approach that improves the final placement position of the workpiece, where IBVS control takes over workpiece placement during the final approach of the robot. Moving object detection is used to automatically locate the positions of an object and a hand in a 2D image space, and then line features on a workpiece are identified by removing line features belonging to the hand using hand keypoint detection. 一种用于基于利用来自相机的图像的人类演示来教导和控制机器人执行操作的方法。该方法包括演示阶段，其中相机检测人手抓取并移动工件以限定工件的机器人移动的粗略轨迹。在演示阶段期间收集的工件上的线特征或其它几何特征用于基于图像的视觉伺服(IBVS)接近，其改进工件的最终放置位置，其中IBVS控制在机器人的最终接近期间接管工件放置。使用移动对象检测来