Deep learning augmented infrared thermography for unmanned aerial vehicles structural health monitoring of roadways

• Published in: Automation in construction Vol. 148; p. 104784
• Main Authors: Kulkarni, Nitin Nagesh, Raisi, Koosha, Valente, Nicholas A., Benoit, Jason, Yu, Tzuyang, Sabato, Alessandro
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2023
• Subjects: Deep learning; Infrared thermography; Roadways; Sparse-principal component thermography; Deep learning; Infrared thermography; Sparse-principal component thermography; Roadways
• ISSN: 0926-5805; 1872-7891
• DOI: 10.1016/j.autcon.2023.104784

Sinkholes form when culverts beneath roadways fail, posing a safety hazard that requires costly repairs and causes traffic delays. Detecting the onset of sub-pavement voids can prevent sudden pavement failure. This research presents an automated approach for increasing the detection accuracy of sub-pavement voids from infrared (IR) images collected with an unmanned aerial vehicle. A framework is developed that relies on i) principal component thermography analysis to increase accuracy of the damage detection and ii) the EfficientDet algorithm to automate the inspection process. Field tests on a roadway demonstrated the capabilities of the proposed framework in detecting sub-pavements voids with accuracy comparable to ground penetrating radar while significantly reducing the inspection time. The automated detection method yielded a mean average precision of 0.85 when detecting both visible and sub-pavement defects, outperforming other deep learning algorithms by 24%. The feasibility of the approach, best practices, and lessons learned are presented. •Novel automated UAV-borne roadway inspection technique using infrared thermography.•Infrared images' signal-to-noise ratio improved using reduced order models.•Evaluation of effects of image sampling rate on thermography accuracy.•Autonomous detection of sub-pavement damages with 85% mean average precision.