Semantic Segmentation of Urban Scenes via Domain Adaptation of SYNTHIA

• Published in: Domain Adaptation in Computer Vision Applications pp. 227 - 241
• Main Authors: Ros, German, Sellart, Laura, Villalonga, Gabriel, Maidanik, Elias, Molero, Francisco, Garcia, Marc, Cedeño, Adriana, Perez, Francisco, Ramirez, Didier, Escobar, Eduardo, Gomez, Jose Luis, Vazquez, David, Lopez, Antonio M.
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2017; Springer International Publishing
• Series: Advances in Computer Vision and Pattern Recognition
• Subjects: Artificial intelligence; Image processing; Public administration
• ISBN: 3319583468; 9783319583464
• ISSN: 2191-6586; 2191-6594
• DOI: 10.1007/978-3-319-58347-1_12

Vision-based semantic Semantic segmentation in urban scenarios is a key functionality for autonomous driving. Recent revolutionary results of deep convolutional neural networks (CNNs) foreshadow the advent of reliable classifiers to perform such visual tasks. However, CNNs require learning of many parameters from raw images; thus, having a sufficient amount of diverse images with class annotations is needed. These annotations are obtained via cumbersome, human labor which is particularly challenging for semantic segmentation since pixel-level annotations are required. In this chapter, we propose to use a combination of a virtual Virtual world to automatically generate realistic synthetic images with pixel-level annotations, and domain adaptation to transfer the models learned to correctly operate in real scenarios. We address the question of how useful synthetic Synthetic data can be for semantic segmentation—Semantic segmentation particular, when using a CNN paradigm. In order to answer this question we have generated a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations and object identifiers. We use SYNTHIA in combination with publicly available real-world urban images with manually provided annotations. Then, we conduct experiments with CNNs that show that combining SYNTHIA with simple domain adaptation techniques in the training stage significantly improves performance on semantic segmentation.