YOLOv5-SFE: An algorithm fusing spatio-temporal features for detecting and recognizing workers' operating behaviors

• Published in: Advanced engineering informatics Vol. 56; p. 101988
• Main Authors: Li, Lijuan, Zhang, Peng, Yang, Shipin, Jiao, Wenhua
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2023
• Subjects: Feature enhancement; Hash sampling algorithm; Spatio-temporal feature fusion; YOLOv5-SFE; Feature enhancement; Spatio-temporal feature fusion; YOLOv5-SFE; Hash sampling algorithm
• ISSN: 1474-0346; 1873-5320
• DOI: 10.1016/j.aei.2023.101988

The occurrence of production accidents can be effectively reduced by monitoring workers' operating behaviors in real time. However, most of the monitoring tasks are currently performed by the monitoring personnel, which takes up a lot of manpower and material resources. To solve this problem, a YOLOv5-SFE algorithm is proposed in this paper for real-time detection and recognition of workers' operating behaviors. The YOLOv5-SFE algorithm makes the following contributions: (1) During data preprocessing, a hash sampling algorithm is used to extract frames with low similarity. (2) A feature enhancement module is designed and integrated into YOLOv5 to distinguish between valid and invalid information. (3) A convolution-based spatio-temporal feature fusion module is designed and is inserted after the extraction of spatial features to extract the temporal features between multiple frames. The videos of workers' operating behaviors are from factories’ industrial scene. The improved algorithm in this paper was trained and tested on the dataset. Compared with the original algorithm, the accuracy of the algorithm improves from 89.3% to 94.7%, the recall improves from 81.5% to 90.8%, and the mean average precision(mAP) improves from 88.2% to 92%. The results show that the improved algorithm is able to accurately detect and recognize workers' operating behaviors in real time, thereby improving the safety of the production process.