Spatial and Color Information-Based Foreign Object Attachment Detection and Device-Foreign Object Segmentation Method

• Published in: 2023 China Automation Congress (CAC) pp. 5386 - 5391
• Main Authors: Li, Bin, Wang, Quan, Jiang, Chao, Li, Xiaolei, Mao, Chunchun, Liu, Chunming, Wang, Siyuan, Sun, Shiying, Hui, Xiaolong
• Format: Conference Proceeding
• Language: English
• Published: IEEE 17.11.2023
• Subjects: 3D point cloud; foreign object attachment detection; Image color analysis; Inspection; intelligent inspection; multimodal information fusion; Object detection; Point cloud compression; substation; Substations; Task analysis; visual localization; Visualization
• ISSN: 2688-0938
• DOI: 10.1109/CAC59555.2023.10450428

This paper proposes a method for foreign object detection and equipment-foreign object segmentation based on spatial and color information, with the context of intelligent inspection in substations. The method addresses the challenge of accurately locating foreign objects, which cannot be reliably achieved using existing image-based approaches, by integrating multi-source sensor information at the edge of the inspection device. The proposed method, when combined with inspection robots, enables automated foreign object removal. The approach starts by capturing multiple frames of three-dimensional point cloud data using point cloud and visual acquisition devices. Subsequently, processes such as stitching, downsampling, and extraction of the main point cloud are performed. The PointN et framework is then utilized to fuse spatial and color modalities of the point cloud to predict the occurrence of foreign object attachments and perform segmentation on the main point cloud. The effectiveness of the proposed method is validated through a case study involving foreign object detection and segmentation on power lines. Compared to detection and segmentation methods based on single-frame data and single position information, the proposed approach achieves higher accuracy in the segmentation task.