A survey on deep learning techniques for image and video semantic segmentation

• Published in: Applied soft computing Vol. 70; pp. 41 - 65
• Main Authors: Garcia-Garcia, Alberto, Orts-Escolano, Sergio, Oprea, Sergiu, Villena-Martinez, Victor, Martinez-Gonzalez, Pablo, Garcia-Rodriguez, Jose
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.09.2018
• Subjects: Deep learning; Scene labeling; Semantic segmentation; Deep learning; Semantic segmentation; Scene labeling
• ISSN: 1568-4946; 1872-9681
• DOI: 10.1016/j.asoc.2018.05.018

•An in-depth review of deep learning methods for semantic segmentation applied to various areas.•An overview of background concepts and formulation for newcomers.•A structured and logical review of datasets and methods, highlighting their contributions and significance.•Quantitative comparison of performance and accuracy on common datasets.•A discussion of future works and promising research lines and conclusions about the state of the art of the field. Image semantic segmentation is more and more being of interest for computer vision and machine learning researchers. Many applications on the rise need accurate and efficient segmentation mechanisms: autonomous driving, indoor navigation, and even virtual or augmented reality systems to name a few. This demand coincides with the rise of deep learning approaches in almost every field or application target related to computer vision, including semantic segmentation or scene understanding. This paper provides a review on deep learning methods for semantic segmentation applied to various application areas. Firstly, we formulate the semantic segmentation problem and define the terminology of this field as well as interesting background concepts. Next, the main datasets and challenges are exposed to help researchers decide which are the ones that best suit their needs and goals. Then, existing methods are reviewed, highlighting their contributions and their significance in the field. We also devote a part of the paper to review common loss functions and error metrics for this problem. Finally, quantitative results are given for the described methods and the datasets in which they were evaluated, following up with a discussion of the results. At last, we point out a set of promising future works and draw our own conclusions about the state of the art of semantic segmentation using deep learning techniques.