Research on Multi-Object Sorting System Based on Deep Learning

• Published in: Sensors (Basel, Switzerland) Vol. 21; no. 18; p. 6238
• Main Authors: Zhang, Hongyan, Liang, Huawei, Ni, Tao, Huang, Lingtao, Yang, Jinsong
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 17.09.2021; MDPI
• Subjects: Accuracy; Algorithms; Cameras; Deep learning; instance segmentation; Machine learning; Neural networks; pose estimation; Principal components analysis; robot sorting; Robots; rotating target detection; Semantics
• DOI: 10.3390/s21186238
• ISSN: 1424-8220

As a complex task, robot sorting has become a research hotspot. In order to enable robots to perform simple, efficient, stable and accurate sorting operations for stacked multi-objects in unstructured scenes, a robot multi-object sorting system is built in this paper. Firstly, the training model of rotating target detection is constructed, and the placement state of five common objects in unstructured scenes is collected as the training set for training. The trained model is used to obtain the position, rotation angle and category of the target object. Then, the instance segmentation model is constructed, and the same data set is made, and the instance segmentation network model is trained. Then, the optimized Mask R-CNN instance segmentation network is used to segment the object surface pixels, and the upper surface point cloud is extracted to calculate the normal vector. Then, the angle obtained by the normal vector of the upper surface and the rotation target detection network is fused with the normal vector to obtain the attitude of the object. At the same time, the grasping order is calculated according to the average depth of the surface. Finally, after the obtained object posture, category and grasping sequence are fused, the performance of the rotating target detection network, the instance segmentation network and the robot sorting system are tested on the established experimental platform. Based on this system, this paper carried out an experiment on the success rate of object capture in a single network and an integrated network. The experimental results show that the multi-object sorting system based on deep learning proposed in this paper can sort stacked objects efficiently, accurately and stably in unstructured scenes.