Recommendations for evaluating the performance of background subtraction algorithms for surveillance systems

• Published in: Multimedia tools and applications Vol. 80; no. 3; pp. 4421 - 4454
• Main Authors: Sanches, Silvio Ricardo Rodrigues, Sementille, Antonio Carlos, Aguilar, Ivan Abdo, Freire, Valdinei
• Format: Journal Article
• Language: English
• Published: New York Springer US 2021; Springer Nature B.V
• Subjects: Algorithms; Computer Communication Networks; Computer Science; Data Structures and Information Theory; Multimedia Information Systems; Performance evaluation; Special Purpose and Application-Based Systems; Statistical analysis; Subtraction; Surveillance; Surveillance systems; Recommendations; Surveillance systems; Background subtraction; Performance assessment
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-020-09838-x

Background subtraction is a prerequisite for a wide range of applications, including video surveillance systems. A significant number of algorithms are often developed and published in different publication mediums in the area, such as workshops, symposiums, conferences, and journals. An important task in presenting a new background subtraction algorithms is to clearly show that its performance outperforms the performance of the state-of-the-art algorithms. In this paper, we present recommendations on how to evaluate the performance of background subtraction algorithms for surveillance systems. We identified, through a systematic mapping, the key steps and components of this evaluation process – procedures, methods, and tools – most used by the authors in each of these steps. Considering this statistical analysis, we perform a theoretical analysis of the most used key components to identify their pros and cons. Then, we define a set of recommendations that aim to standardize and clarify the performance evaluation process of a new background subtraction algorithm.