Automated wall‐climbing robot for concrete construction inspection

• Published in: Journal of field robotics Vol. 40; no. 1; pp. 110 - 129
• Main Authors: Yang, Liang, Li, Bing, Feng, Jinglun, Yang, Guoyong, Chang, Yong, Jiang, Biao, Xiao, Jizhong
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc 01.01.2023
• Subjects: 3D robotic mapping; Area; artificial intelligence; Artificial neural networks; Building codes; Climbing; Concrete bridges; Concrete construction; Concrete structures; Construction inspection; Cracks; Data collection; Finite element method; Flaw detection; GPS denied; Image reconstruction; Inertial navigation; Inertial platforms; inspection robots; Modules; Navigation systems; Robots; Semantics; Surface defects; Three dimensional models
• DOI: 10.1002/rob.22119
• ISSN: 1556-4959

Human‐made concrete structures require cutting‐edge inspection tools to ensure the quality of the construction to meet the applicable building codes and to maintain the sustainability of the aging infrastructure. This paper introduces a wall‐climbing robot for metric concrete inspection that can reach difficult‐to‐access locations with a close‐up view for visual data collection and real‐time flaws detection and localization. The wall‐climbing robot is able to detect concrete surface flaws (i.e., cracks and spalls) and produce a defect‐highlighted 3D model with extracted location clues and metric measurements. The system encompasses four modules, including a data collection module to capture RGB‐D frames and inertial measurement unit data, a visual–inertial navigation system module to generate pose‐coupled keyframes, a deep neural network module (namely InspectionNet) to classify each pixel into three classes (background, crack, and spall), and a semantic reconstruction module to integrate per‐frame measurement into a global volumetric model with defects highlighted. We found that commercial RGB‐D camera output depth is noisy with holes, and a Gussian‐Bilateral filter for depth completion is introduced to inpaint the depth image. The method achieves the state‐of‐the‐art depth completion accuracy even with large holes. Based on the semantic mesh, we introduce a coherent defect metric evaluation approach to compute the metric measurement of crack and spall area (e.g., length, width, area, and depth). Field experiments on a concrete bridge demonstrate that our wall‐climbing robot is able to operate on a rough surface and can cross over shallow gaps. The robot is capable to detect and measure surface flaws under low illuminated environments and texture‐less environments. Besides the robot system, we create the first publicly accessible concrete structure spalls and cracks data set that includes 820 labeled images and over 10,000 field‐collected images and release it to the research community.