Estimation of pavement crack ratio by top-view transformation of in-vehicle smartphone camera and deep learning-based classification

• Published in: Artificial Intelligence and Data Science Vol. 3; no. 3; pp. 26 - 39
• Main Authors: GEDA, Jose Maria Guyamin, XUE, Kai, NAGAYAMA, Tomonori, ZHAO, Boyu, NAKA, Michihiro
• Format: Journal Article
• Language: English
• Published: Japan Society of Civil Engineers 2022
• Subjects: crack ratio; deep learning; image stitching; road condition assessment; top-view transformation
• ISSN: 2435-9262
• DOI: 10.11532/jsceiii.3.3_26

The conventional road crack ratio evaluation in Japan uses top-view images of pavement obtained from precision line scan cameras and manually classifies 0.5m grids of pavement surface based on the number of cracks. While inexpensive crack evaluation using in-vehicle smartphone cameras has been proposed, previous work cannot calculate the crack ratio based on the index definition and have limited accuracy. This paper proposes a vision-based top-view transformation and image stitching algorithm for road crack ratio evaluation using video captured by an in-vehicle smartphone camera. Four conditions are used to perform the parameter calibration: 1) horizontal manhole axis, 2) parallel lane line, 3) circular manhole, 4) vertical lane line conditions. After the successful top-view transformation, feature matching is conducted to pairs of successive frames to calculate the homography matrix between the two images, which is used for the image stitching of successive frames and obtaining the translation offset between the images. Based on the calculated translation offset and the extracted frame distance interval, the pixel-to-real-distance conversion factor is calculated. The image is divided into a 0.5m grid based on the index definition. An image classification model was trained to classify each grid box according to the number of cracks. The results showed that: 1) a fine-resolution image of road top-view can be produced from successive images captured by an in-vehicle smartphone camera, and 2) the crack ratio can be accurately estimated from these images automatically.