Real-Time Semantic Segmentation via Spatial-Detail Guided Context Propagation

• Published in: IEEE transaction on neural networks and learning systems Vol. 36; no. 3; pp. 4042 - 4053
• Main Authors: Hao, Shijie, Zhou, Yuan, Guo, Yanrong, Hong, Richang, Cheng, Jun, Wang, Meng
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.03.2025
• Subjects: Accuracy; Computational modeling; contextual information; Convolution; Costs; deep learning; Image segmentation; Pipelines; Real-time systems; semantic segmentation; Semantics; speed
• ISSN: 2162-237X; 2162-2388; 2162-2388
• DOI: 10.1109/TNNLS.2022.3154443

Nowadays, vision-based computing tasks play an important role in various real-world applications. However, many vision computing tasks, e.g., semantic segmentation, are usually computationally expensive, posing a challenge to the computing systems that are resource-constrained but require fast response speed. Therefore, it is valuable to develop accurate and real-time vision processing models that only require limited computational resources. To this end, we propose the spatial-detail guided context propagation network (SGCPNet) for achieving real-time semantic segmentation. In SGCPNet, we propose the strategy of spatial-detail guided context propagation. It uses the spatial details of shallow layers to guide the propagation of the low-resolution global contexts, in which the lost spatial information can be effectively reconstructed. In this way, the need for maintaining high-resolution features along the network is freed, therefore largely improving the model efficiency. On the other hand, due to the effective reconstruction of spatial details, the segmentation accuracy can be still preserved. In the experiments, we validate the effectiveness and efficiency of the proposed SGCPNet model. On the Cityscapes dataset, for example, our SGCPNet achieves 69.5% mIoU segmentation accuracy, while its speed reaches 178.5 FPS on 768 <inline-formula> <tex-math notation="LaTeX">\times </tex-math></inline-formula> 1536 images on a GeForce GTX 1080 Ti GPU card. In addition, SGCPNet is very lightweight and only contains 0.61 M parameters. The code will be released at https://github.com/zhouyuan888888/SGCPNet .