Vision-Based Guiding System for Autonomous Robotic Corner Cleaning of Window Frames

• Published in: Buildings (Basel) Vol. 13; no. 12; p. 2990
• Main Authors: Tung, Tzu-Jan, Al-Hussein, Mohamed, Martinez, Pablo
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.12.2023
• Subjects: Adaptability; Aluminum; Artificial intelligence; Assembly lines; Automation; Cleaning; Cleaning process; Composite materials; Cutting equipment; Edge detection; Equipment and supplies; Image processing; Industrial applications; Industry 4.0; Information processing; Machine vision; Manufacturers; Manufacturing; Mechanization; Modules; offsite construction; Quality control; Robot arms; Robotics; Robotics industry; Robots; Seam welds; vision-guided robotics; Welding; window manufacturing; Workers
• ISSN: 2075-5309; 2075-5309
• DOI: 10.3390/buildings13122990

Corner cleaning is the most important manufacturing step of window framing to ensure aesthetic quality. After the welding process, the current methods to clean the welding seams lack quality control and adaptability. This increases rework, cost, and the waste produced in manufacturing and is largely due to the use of CNC cutting machines, as well as the reliance on manual inspection and weld seam cleaning. Dealing with manufacturing imperfections becomes a challenging task, as CNC machines rely on predetermined cleaning paths and frame information. To tackle such challenges using Industry 4.0 approaches and automation technology, such as robots and sensors, in this paper, a novel intelligent system is proposed to increase the process capacity to adapt to variability in weld cleaning conditions while ensuring quality through a combined approach of robot arms and machine vision that replaces the existing manual-based methods. Using edge detection to identify the window position and its orientation, artificial intelligence image processing techniques (Mask R-CNN model) are used to detect the window weld seam and to guide the robot manipulator in its cleaning process. The framework is divided into several modules, beginning with the estimation of a rough position for the purpose of guiding the robot toward the window target, followed by an image processing and detection module used in conjunction with instance segmentation techniques to segment the target area of the weld seam, and, finally, the generation of cleaning paths for further robot manipulation. The proposed robotic system is validated two-fold: first, in a simulated environment and then, in a real-world scenario, with the results obtained demonstrating the effectiveness and adaptability of the proposed system. The evaluation of the proposed framework shows that the trained Mask R-CNN can locate and quantify weld seams with 95% mean average precision (less than 1 cm).