Video segmentation of industrial smoke based on dynamic fully convolutional network-Gaussian mixture model and multi-scale fusion attention module

• Published in: Journal of electronic imaging Vol. 32; no. 3; p. 033038
• Main Authors: Wenyu, Ding, Hui, Liu, Fugang, Chen
• Format: Journal Article
• Language: English
• Published: Society of Photo-Optical Instrumentation Engineers 01.05.2023; SPIE
• Subjects: Gaussian processes; Image processing; Neural networks; convolutional network; spatiotemporal feature; industrial smoke; attention module; Gaussian mixture model; multi-scale fusion
• DOI: 10.1117/1.JEI.32.3.033038
• ISSN: 1017-9909

Accurately segmenting industrial smoke in videos plays a crucial role in assessing pollution levels based on smoke image evaluation. However, existing fully convolutional networks (FCNs) face challenges in precisely segmenting the edges of industrial smoke and exhibit low extraction and segmentation accuracy for small target smoke. To address this issue, we propose a video segmentation method specifically designed for industrial smoke. This method utilizes the dynamic FCN-Gaussian mixture model (GMM) along with a multi-scale fusion module and an attention module. The FCN-GMM effectively extracts dynamic feature information from spatiotemporal data, capturing motion in video or image sequences while preserving spatial details. The key innovation of FCN-GMM lies in integrating dynamic and static networks through a neural network, enabling the capture of features in both the temporal and spatial domains. Our approach begins by constructing a dynamic feature extraction network that captures spatial and temporal feature information separately during the training process, thereby enhancing the extraction of smoke edges. Additionally, we introduce a mechanism for multi-scale feature fusion and an attention module to effectively extract information related to small target smoke. Our experimental results demonstrate that our network accurately segments significant target smoke compared with FCNs. Furthermore, the network prioritizes the consideration of smoke edge information and improves the extraction of small target smoke, thereby enhancing the overall accuracy of smoke image segmentation with an increase of up to 10% in the intersection over union index.