Generating real-world-like labelled synthetic datasets for construction site applications

• Published in: Automation in construction Vol. 151; p. 104850
• Main Authors: Barrera-Animas, Ari Yair, Davila Delgado, Juan Manuel
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.07.2023
• Subjects: Auto-annotation; Construction field; Object detection; Synthetic dataset; Auto-annotation; Synthetic dataset; Construction field; Object detection
• ISSN: 0926-5805; 1872-7891
• DOI: 10.1016/j.autcon.2023.104850

Having synthetic image generation and automatic labelling as two separate processes remains one of the main limitations of automatic generation of large real-world synthetic datasets. To overcome this drawback, a methodology to perform both tasks in a simultaneous and automatic manner is proposed. To resemble real-world scenarios, a diverse set of rendering configurations of illumination, locations, and sizes are presented. For testing, three synthetic datasets (S, M and SM), oriented to the construction field, were generated. Faster R-CNN, RetinaNet, and YoloV4 detection algorithms were used to independently evaluate the datasets using the COCO evaluation metrics and the PascalVOC Mean Average Accuracy metric. Results show that, in general, the S dataset performed 1.2% better in the evaluation metrics and that the SM dataset obtained better plots of training and validation loss curves in each detector; highlighting the combinational usage of images with single and multiple objects as a better generalisation approach. •Approach to automatically generate and label real-world synthetic datasets.•Object detection of workers and machinery using synthetic datasets.•Use of real-world conditions to automatically build labelled synthetic datasets.•YoloV4, RetinaNet and Faster R-CNN trained with real-world synthetic datasets.•Datasets of single and multiple objects generalise and resemble real-world scenes.