Concrete Bridge Defects Identification and Localization Based on Classification Deep Convolutional Neural Networks and Transfer Learning

• Published in: Remote sensing (Basel, Switzerland) Vol. 14; no. 19; p. 4882
• Main Authors: Zoubir, Hajar, Rguig, Mustapha, El Aroussi, Mohamed, Chehri, Abdellah, Saadane, Rachid, Jeon, Gwanggil
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.10.2022
• Subjects: Accuracy; Algorithms; Artificial neural networks; Automation; Back propagation; Back propagation networks; Bridge inspection; Classification; Concrete; concrete bridge; Concrete bridges; Cracks; Damage detection; Datasets; deep convolutional neural network; Deep learning; defect; Defects; Efflorescence; Image annotation; Image processing; Image segmentation; Information processing; interpretation techniques; Learning; Localization; Machine learning; Massive data points; Neural networks; Performance evaluation; Remote sensing; Semantics; Sensors; Spalling; Transfer learning; Unmanned aerial vehicles; visual inspection
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs14194882

Conventional practices of bridge visual inspection present several limitations, including a tedious process of analyzing images manually to identify potential damages. Vision-based techniques, particularly Deep Convolutional Neural Networks, have been widely investigated to automatically identify, localize, and quantify defects in bridge images. However, massive datasets with different annotation levels are required to train these deep models. This paper presents a dataset of more than 6900 images featuring three common defects of concrete bridges (i.e., cracks, efflorescence, and spalling). To overcome the challenge of limited training samples, three Transfer Learning approaches in fine-tuning the state-of-the-art Visual Geometry Group network were studied and compared to classify the three defects. The best-proposed approach achieved a high testing accuracy (97.13%), combined with high F1-scores of 97.38%, 95.01%, and 97.35% for cracks, efflorescence, and spalling, respectively. Furthermore, the effectiveness of interpretable networks was explored in the context of weakly supervised semantic segmentation using image-level annotations. Two gradient-based backpropagation interpretation techniques were used to generate pixel-level heatmaps and localize defects in test images. Qualitative results showcase the potential use of interpretation maps to provide relevant information on defect localization in a weak supervision framework.