A Transformer-based Video Segmentation Method for Road Weather Conditions

Detecting road weather conditions is very important for autonomous driving systems. Many existing studies use image classification or segmentation methods to recognize road weather conditions, but one single image can be easily affected by environmental factors such as lighting, resulting in lower i...

Full description

Saved in:
Bibliographic Details
Published in2024 International Symposium on Intelligent Robotics and Systems (ISoIRS) pp. 299 - 304
Main Authors Guo, Yu'Ang, Yu, Guizhen, Zhang, Chuang
Format Conference Proceeding
LanguageEnglish
Published IEEE 14.06.2024
Subjects
Computer vision
Environmental factors
Image segmentation
Real-time systems
Road weather condition
Roads
Robustness
Transformer
Transformers
Video segmentation
Video sequences
Online AccessGet full text

Cover

More Information
Summary:Detecting road weather conditions is very important for autonomous driving systems. Many existing studies use image classification or segmentation methods to recognize road weather conditions, but one single image can be easily affected by environmental factors such as lighting, resulting in lower image quality and false detections. In this paper, we propose a video segmentation method to detect dry, wet and snowy road areas. By fusing the features of consecutive image frames in the video, the impact of poor quality images on the detection results can be reduced, improving the robustness of the algorithm. Meanwhile, we propose the Temporal Feature Fusion Block (TFFB) based on the Transformer decoder to achieve the feature-level fusion, avoiding the image registration required for traditional temporal image fusion. This reduces the algorithm complexity and improves real-time performance. In order to verify the effectiveness of the algorithm, we constructed a dataset of road weather conditions by collecting images and manually annotated them at the pixel level. The dataset has 12 video sequences with a total of about 5000 frames. Experimental results show that the mean Intersection over Union (mIoU) of road weather condition segmentation can be 87.62% by multiple frame feature fusion, which confirms the effectiveness of the algorithm.
DOI:10.1109/ISoIRS63136.2024.00065