Segmentation of Low-Level Temporal Plume Patterns From IR Video

• Published in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) pp. 847 - 854
• Main Authors: Bhatt, Rajeev, Uzunbas, M. Gokhan, Hoang, Thai, Whiting, Ozge C.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.06.2019
• Subjects: Cameras; Decoding; Feature extraction; Kernel; Motion segmentation; Task analysis; Three-dimensional displays
• ISSN: 2160-7516
• DOI: 10.1109/CVPRW.2019.00113

In this paper, a method to segment out gas or steam plumes in IR videos collected from fixed cameras is presented. We propose a spatio-temporal U-Net architecture that captures deforming blobs of gas/steam plumes that have a unique temporal signature. In this task, the blob shapes are not semantically meaningful and change from frame to frame with no consistency across different exemplar plumes; however, there is spatial and temporal continuity in the way blobs deform suggesting a need for a low-level spatio-temporal segmentation network. The proposed method is compared to an LSTM-based segmentation network on a challenging IR video dataset collected in a controlled environment. In the controlled dataset there is motion due to steam plumes with deforming blob patterns as well as due to walking people with more structured high-level patterns. The experiments show that plume patterns are successfully segmented out with no confusion to moving people and the proposed spatiotemporal U-Net outperforms LSTM-based network in terms of pixelwise accuracy of output masks.