Bilayer Segmentation of Live Video in Uncontrolled Environments for Background Substitution: An Overview and Main Challenges

• Published in: Revista IEEE América Latina Vol. 10; no. 5; pp. 2138 - 2149
• Main Authors: Sanches, S. R. R., Nakamura, R., da Silva, V. F., Tori, R.
• Format: Journal Article
• Language: English
• Published: Los Alamitos IEEE 01.09.2012; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Background Substitution; Bilayer Segmentation; Cameras; Color; Computer Vision; Devices; Illumination; Image coding; Image Processing; Image segmentation; Live Video; Optimization; Probabilistic logic; Robustness; Segmentation; Solid modeling; Studies; Tasks
• ISSN: 1548-0992; 1548-0992
• DOI: 10.1109/TLA.2012.6362359

Bilayer segmentation of live video in uncontrolled environments is an essential task for home applications in which the original background of the scene must be replaced, as in video chats or traditional videoconference. The main challenge in such conditions is overcome all difficulties in problem-situations (e.g., illumination change, distract events such as element moving in the background and camera shake) that may occur while the video is being captured. This paper presents a survey of segmentation methods for background substitution applications, describes the main concepts and identifies events that may cause errors. Our analysis shows that although robust methods rely on specific devices (multiple cameras or sensors to generate depth maps) which aid the process. In order to achieve the same results using conventional devices (monocular video cameras), most current research relies on energy minimization frameworks, in which temporal and spacial information are probabilistically combined with those of color and contrast.