A deep convolutional neural network for video sequence background subtraction

• Published in: Pattern recognition Vol. 76; pp. 635 - 649
• Main Authors: Babaee, Mohammadreza, Dinh, Duc Tung, Rigoll, Gerhard
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.04.2018
• Subjects: Background subtraction; CNN; Deep learning; Video segmentation; Deep learning; Video segmentation; CNN; Background subtraction
• ISSN: 0031-3203; 1873-5142
• DOI: 10.1016/j.patcog.2017.09.040

•We propose a novel approach based on deep learning for background subtraction from video sequences.•A new algorithm to generate background model has been proposed.•Input image patches and their corresponding background images are fed into CNN to do background subtraction.•We utilized median filter to enhance the segmentation results.•Experiments of Change detection results confirm the performance of the proposed approach. In this work, we present a novel background subtraction from video sequences algorithm that uses a deep Convolutional Neural Network (CNN) to perform the segmentation. With this approach, feature engineering and parameter tuning become unnecessary since the network parameters can be learned from data by training a single CNN that can handle various video scenes. Additionally, we propose a new approach to estimate background model from video sequences. For the training of the CNN, we employed randomly 5% video frames and their ground truth segmentations taken from the Change Detection challenge 2014 (CDnet 2014). We also utilized spatial-median filtering as the post-processing of the network outputs. Our method is evaluated with different data-sets, and it (so-called DeepBS) outperforms the existing algorithms with respect to the average ranking over different evaluation metrics announced in CDnet 2014. Furthermore, due to the network architecture, our CNN is capable of real time processing.