A Recurrent Adaptive Network: Balanced Learning for Road Crack Segmentation with High-Resolution Images

• Published in: Remote sensing (Basel, Switzerland) Vol. 14; no. 14; p. 3275
• Main Authors: Zhang, Yi, Fan, Junfu, Zhang, Mengzhen, Shi, Zongwen, Liu, Rufei, Guo, Bing
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.07.2022
• Subjects: adaptive loss; Annotations; Artificial neural networks; balanced deep learning; Cameras; convolutional neural networks; Cracks; Datasets; Deep learning; High resolution; Image processing; Image resolution; Image segmentation; Inspection; Machine learning; Neural networks; Performance evaluation; Recall; Remote sensing; road crack segmentation; Road maintenance; Roads; Roads & highways; Semantics; Thermodynamics; Training; Unmanned aerial vehicles; Wavelet transforms
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs14143275

Road crack segmentation based on high-resolution images is an important task in road service maintenance. The undamaged road surface area is much larger than the damaged area on a highway. This imbalanced situation yields poor road crack segmentation performance for convolutional neural networks. In this paper, we first evaluate the mainstream convolutional neural network structure in the road crack segmentation task. Second, inspired by the second law of thermodynamics, an improved method called a recurrent adaptive network for a pixelwise road crack segmentation task is proposed to solve the extreme imbalance between positive and negative samples. We achieved a flow between precision and recall, similar to the conduction of temperature repetition. During the training process, the recurrent adaptive network (1) dynamically evaluates the degree of imbalance, (2) determines the positive and negative sampling rates, and (3) adjusts the loss weights of positive and negative features. By following these steps, we established a channel between precision and recall and kept them balanced as they flow to each other. A dataset of high-resolution road crack images with annotations (named HRRC) was built from a real road inspection scene. The images in HRRC were collected on a mobile vehicle measurement platform by high-resolution industrial cameras and were carefully labeled at the pixel level. Therefore, this dataset has sufficient data complexity to objectively evaluate the real performance of convolutional neural networks in highway patrol scenes. Our main contribution is a new method of solving the data imbalance problem, and the method of guiding model training by analyzing precision and recall is experimentally demonstrated to be effective. The recurrent adaptive network achieves state-of-the-art performance on this dataset.