Deep leaf‐bootstrapping generative adversarial network for structural image data augmentation

• Published in: Computer-aided civil and infrastructure engineering Vol. 34; no. 9; pp. 755 - 773
• Main Authors: Gao, Yuqing, Kong, Boyuan, Mosalam, Khalid M.
• Format: Journal Article
• Language: English
• Published: Hoboken Wiley Subscription Services, Inc 01.09.2019
• Subjects: Affine transformations; Damage detection; Data augmentation; Data management; Generative adversarial networks; Identification methods; Image enhancement; Image quality; Machine learning; Performance enhancement
• ISSN: 1093-9687; 1467-8667
• DOI: 10.1111/mice.12458

Employing Deep Learning (DL) technologies to solve Civil Engineering problems is an emerging topic in recent years. However, due to the lack of labeled data, it is difficult to obtain accurate results with DL. One commonly used method to tackle this issue is to use affine transformation to augment the data set, but it can only generate new images that are highly correlated with the original ones. Moreover, unlike normal natural objects, distribution of structural images is much more complex and mixed. To address these challenges, Generative Adversarial Network (GAN) can be one feasible choice. We introduce one specific generative model, namely, Deep Convolutional Generative Adversarial Network (DCGAN) and propose a Leaf‐Bootstrapping (LB) method to improve the performance of this DCGAN. To effectively and quantitatively evaluate the quality of the synthetic images generated by DCGAN to complement human evaluation, Self‐Inception Score (SIS) and Generalization Ability (GA) are proposed. We also propose a pipeline based on Transfer Learning (TL) using synthetic images to help enhance a weak classifier performance under the condition of low‐data regime and limited computational resources. Finally, we conduct computer experiments with the proposed methods for two scenarios (scene level identification and damage state check) and one special synthetic data aggregation case. The results demonstrate the effectiveness and robustness of the proposed methods.