A simulation‐assisted point cloud segmentation neural network for human–robot interaction applications

• Published in: Journal of field robotics Vol. 41; no. 8; pp. 2689 - 2704
• Main Authors: Lin, Jingxin, Zhong, Kaifan, Gong, Tao, Zhang, Xianmin, Wang, Nianfeng
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc 01.12.2024
• Subjects: human–robot interaction; Image segmentation; neural network; Neural networks; point cloud segmentation; prior information; Robots; simulation assistance
• ISSN: 1556-4959; 1556-4967
• DOI: 10.1002/rob.22385

With the advancement of industrial automation, the frequency of human–robot interaction (HRI) has significantly increased, necessitating a paramount focus on ensuring human safety throughout this process. This paper proposes a simulation‐assisted neural network for point cloud segmentation in HRI, specifically distinguishing humans from various surrounding objects. During HRI, readily accessible prior information, such as the positions of background objects and the robot's posture, can generate a simulated point cloud and assist in point cloud segmentation. The simulation‐assisted neural network utilizes simulated and actual point clouds as dual inputs. A simulation‐assisted edge convolution module in the network facilitates the combination of features from the actual and simulated point clouds, updating the features of the actual point cloud to incorporate simulation information. Experiments of point cloud segmentation in industrial environments verify the efficacy of the proposed method.