Research on gesture segmentation method based on FCN combined with CBAM-ResNet50

• Published in: Signal, image and video processing Vol. 18; no. 11; pp. 7729 - 7740
• Main Authors: Zhao, Hongtu, Liang, Menghua, Li, Hao
• Format: Journal Article
• Language: English
• Published: London Springer London 2024; Springer Nature B.V
• Subjects: Algorithms; Computer Imaging; Computer Science; Feature extraction; Gesture recognition; Image enhancement; Image processing; Image Processing and Computer Vision; Image segmentation; Multimedia Information Systems; Original Paper; Pattern Recognition and Graphics; Signal,Image and Speech Processing; Trunk networks; Vision; FCN; Gesture segmentation; CBAM; Complex backgrounds; ResNet-50
• ISSN: 1863-1703; 1863-1711
• DOI: 10.1007/s11760-024-03423-7

As a key step of gesture recognition, gesture segmentation can effectively reduce the impact of complex backgrounds on recognition results and improve the accuracy of gesture recognition. The gesture segmentation algorithm based on image processing is easily affected by the complex backgrounds in the image, resulting in poor gesture segmentation effect and low precision. To improve the effect of gesture segmentation under complex backgrounds and increase the accuracy of gesture recognition systems, this paper proposes a gesture segmentation method based on FCN combined with CBAM-ResNet50 network model. The trunk network of FCN is proposed as a new ResNet-50 framework. CBAM attention mechanism is introduced into the residual structure to enhance the extraction ability of multi-scale context information. And we achieve deep feature and shallow feature extraction and fusion by combining expansion convolution and enhancing the parameters of the convolutional layer, so as to improve the precision of gesture segmentation. In addition, the methods of data preprocessing and loading pre-training weights are used to enhance the model’s robustness and performance and expedite the training time. In the experiments, the NUS-II gesture data set with a complex background was used for testing. The average recognition rate of gesture pixels was 97.41% and gesture IoU was 94.54%. The experimental results show that the proposed method has a good effect on gesture segmentation for gesture images with complex backgrounds.