IAN: Instance-Augmented Net for 3D Instance Segmentation

• Published in: IEEE robotics and automation letters Vol. 8; no. 7; pp. 4354 - 4361
• Main Authors: Wan, Zihao, Hu, Jianhua, Zhang, Haojian, Wang, Yunkuan
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 01.07.2023; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Aggregates; Clutter; data sets for robotic vision; Deep learning for visual perception; Feature extraction; Feature maps; Grasping (robotics); Instance segmentation; Noise measurement; Occlusion; Point cloud compression; RGB-D perception; Semantics; Solid modeling; Three-dimensional displays
• ISSN: 2377-3766; 2377-3766
• DOI: 10.1109/LRA.2023.3281905

When aggregating local information from neighbors, prevailing 3D instance segmentation backbones only leverage 3D coordinates to find neighboring points without identifying whether these points are from the same object as the query point, which causes the model to gather excessive noisy features. Besides, traditional backbones fail to fully utilize multi-resolution information. Therefore, previous methods have difficulty in segmenting targets in cluttered scenes. To tackle these issues, we propose Instance-Augmented Net (IAN). The keys to our approach are Instance-Augmented Block (IAB), Instance-Augmented Upsampler (IAU), and Attentive Fusion (AF). In IAB, for each foreground point, we leverage its instance information to filter out noisy neighbors from other objects. We also propose IAU to apply this instance-augmented strategy to the upsampling process. Furthermore, to retain comprehensive information, we upsample multi-resolution feature maps and adopt attention generated by AF to fuse them. Notably, by encoding neighborhood information, AF can generate attention at point-level adaptively. Moreover, to further test the generality of models, we present Clutter and Occlusion (CAO), a new 3D instance segmentation dataset tailored for robotic grasping tasks. Extensive experiments on S3DIS, ScanNet and CAO show the effectiveness of our IAN.