Mask-Pose Cascaded CNN for 2D Hand Pose Estimation From Single Color Image

• Published in: IEEE transactions on circuits and systems for video technology Vol. 29; no. 11; pp. 3258 - 3268
• Main Authors: Wang, Yangang, Peng, Cong, Liu, Yebin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.11.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Artificial neural networks; cascaded CNN; Color; Color imagery; Convolutional neural networks; Datasets; Hand pose estimation; Heating systems; Image segmentation; mask prediction; Neural networks; Pose estimation; Three-dimensional displays; Two dimensional displays
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2018.2879980

We present a cascaded convolutional neural network for 2D hand pose estimation from single in-the-wild RGB images. Inspired by the commonly used silhouette information in the generative pose estimation approaches, we build the cascaded network with two stages, including mask prediction stage as well as pose estimation stage. We find that the two stages network architecture for end-to-end training could benefit from each other for detecting the hand mask and 2D pose. To further improve the hand pose detection accuracy, we contribute a new RGB hand dataset named OneHand10K, which contains 10K RGB images. Each image contains one single hand. We manually obtain the segmented mask and labeled keypoints for guided learning. We hope that this dataset will be a benchmark and encourage more people to conduct research on this challenging topic. Experiments on the validation dataset have demonstrated the superior performance of the proposed cascaded convolutional neural network.