Jointly network image processing: multi-task image semantic segmentation of indoor scene based on CNN

• Published in: IET image processing Vol. 14; no. 15; pp. 3689 - 3697
• Main Authors: Huang, Li, He, Meiling, Tan, Chong, Jiang, Du, Li, Gongfa, Yu, Hui
• Format: Journal Article
• Language: English
• Published: The Institution of Engineering and Technology 15.12.2020
• Subjects: complex indoor environment; complex visual tasks; computer vision; different semantic category labels; feature extraction; image classification; image processing; image recognition; image representation; image segmentation; image semantic segmentation network framework; image sensors; indoor scene data; joint target detection; jointly network image processing; learning (artificial intelligence); multitask image semantic segmentation; multivision task; object classification; object detection; position information; research hotspot; robot vision; semantic category information; semantic segmentation branches; Special Section: Evolutionary Computation for Image Processing; target detection network; target tracking; image classification; object classification; position information; object detection; different semantic category labels; semantic segmentation branches; indoor scene data; semantic category information; feature extraction; image segmentation; image semantic segmentation network framework; jointly network image processing; learning (artificial intelligence); image processing; robot vision; research hotspot; joint target detection; image sensors; target detection network; multivision task; target tracking; computer vision; image representation; complex visual tasks; complex indoor environment; multitask image semantic segmentation; image recognition
• ISSN: 1751-9659; 1751-9667
• DOI: 10.1049/iet-ipr.2020.0088

Image semantic segmentation has always been a research hotspot in the field of robots. Its purpose is to assign different semantic category labels to objects by segmenting different objects. However, in practical applications, in addition to knowing the semantic category information of objects, robots also need to know the position information of objects to complete more complex visual tasks. Aiming at a complex indoor environment, this study designs an image semantic segmentation network framework of joint target detection. Using the parallel operation of adding semantic segmentation branches to the target detection network, it innovatively implements multi-vision task combining object classification, detection and semantic segmentation. By designing a new loss function, adjusting the training using the idea of transfer learning, and finally verifying it on the self-built indoor scene data set, the experiment proves that the method in this study is feasible and effective, and has good robustness.