Vision-based multi-level synthetical evaluation of seismic damage for RC structural components: a multi-task learning approach

• Published in: Earthquake Engineering and Engineering Vibration Vol. 22; no. 1; pp. 69 - 85
• Main Authors: Xu, Yang, Qiao, Weidong, Zhao, Jin, Zhang, Qiangqiang, Li, Hui
• Format: Journal Article
• Language: English
• Published: Berlin/Heidelberg Springer Berlin Heidelberg 01.01.2023; Springer Nature B.V; Key Lab of Structures Dynamics Behavior and Control of the Ministry of Education,Harbin Institute of Technology,Harbin 150090,China; School of Civil Engineering,Harbin Institute of Technology,Harbin 150090,China%School of Civil Engineering and Mechanics,Lanzhou University,Lanzhou 730000,China; Key Lab of Smart Prevention and Mitigation of Civil Engineering Disasters of the Ministry of Industry and Information Technology,Harbin Institute of Technology,Harbin 150090,China
• Subjects: Ablation; Civil Engineering; Coders; Coefficients; Comparative analysis; Comparative studies; Components; Computer networks; Computer vision; Computer Vision Empowering Earthquake Engineering and Engineering Vibration; Control; Deep learning; Deterioration; Dynamical Systems; Earth and Environmental Science; Earth Sciences; Earthquake damage; Earthquakes; Encoders-Decoders; Geotechnical Engineering & Applied Earth Sciences; Inspection; Seismic activity; Semantic segmentation; Vibration; seismic damage recognition; multi-task learning; post-earthquake evaluation; structural component segmentation; computer vision; deterioration state assessment
• ISSN: 1671-3664; 1993-503X
• DOI: 10.1007/s11803-023-2153-4

Recent studies for computer vision and deep learning-based, post-earthquake inspections on RC structures mainly perform well for specific tasks, while the trained models must be fine-tuned and re-trained when facing new tasks and datasets, which is inevitably time-consuming. This study proposes a multi-task learning approach that simultaneously accomplishes the semantic segmentation of seven-type structural components, three-type seismic damage, and four-type deterioration states. The proposed method contains a CNN-based encoder-decoder backbone subnetwork with skip-connection modules and a multi-head, task-specific recognition subnetwork. The backbone subnetwork is designed to extract multi-level features of post-earthquake RC structures. The multi-head, task-specific recognition subnetwork consists of three individual self-attention pipelines, each of which utilizes extracted multi-level features from the backbone network as a mutual guidance for the individual segmentation task. A synthetical loss function is designed with real-time adaptive coefficients to balance multi-task losses and focus on the most unstably fluctuating one. Ablation experiments and comparative studies are further conducted to demonstrate their effectiveness and necessity. The results show that the proposed method can simultaneously recognize different structural components, seismic damage, and deterioration states, and that the overall performance of the three-task learning models gains general improvement when compared to all single-task and dual-task models.