Deep learning-based automatic volumetric damage quantification using depth camera

• Published in: Automation in construction Vol. 99; pp. 114 - 124
• Main Authors: Beckman, Gustavo H., Polyzois, Dimos, Cha, Young-Jin
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.03.2019; Elsevier BV
• Subjects: Artificial neural networks; Concrete; Concrete spalling; Convolutional neural network; Damage detection; Deep learning; Depth sensor; Identification methods; Sensors; Spalling; Structural members; Volume quantification; Deep learning; Volume quantification; Concrete spalling; Convolutional neural network; Depth sensor
• ISSN: 0926-5805; 1872-7891
• DOI: 10.1016/j.autcon.2018.12.006

A depth camera or 3-dimensional scanner was used as a sensor for traditional methods to quantify the identified concrete spalling damage in terms of volume. However, to quantify the concrete spalling damage automatically, the first step is to detect (i.e., identify) the concrete spalling. The multiple spots of spalling can be possible within a single structural element or in multiple structural elements. However, there is, as of yet, no method to detect concrete spalling automatically using deep learning methods. Therefore, in this paper, a faster region-based convolutional neural network (Faster R-CNN)-based concrete spalling damage detection method is proposed with an inexpensive depth sensor to quantify multiple instances of spalling simultaneously in the same surface separately and consider multiple surfaces in structural elements. A database composed of 1091 images (with 853 × 1440 pixels) labeled for volumetric damage is developed, and the deep learning network is then modified, trained, and validated using the proposed database. The damage quantification is automatically performed by processing the depth data, identifying surfaces, and isolating the damage after merging the output from the Faster R-CNN with the depth stream of the sensor. The trained Faster R-CNN presented an average precision (AP) of 90.79%. Volume quantifications show a mean precision error (MPE) of 9.45% when considering distances from 100 cm to 250 cm between the element and the sensor. Also, an MPE of 3.24% was obtained for maximum damage depth measurements across the same distance range. •The first method of an automatic concrete spalling volumetric quantification without limitations of setup of a depth camera•An advanced deep learning approach integrated with a structural surface fitting algorithm for automation•The proposed method can quantify multiple volumetric sites of damage in the same structural surface and on multiple surfaces•Fully automated damage detection, localization, and quantification realized by automatically identifying the damaged surface•The proposed method can be used as a prototype for the automatic quantification of damage using other type of depth camera