Weakly Supervised Segmentation Guided Hand Pose Estimation During Interaction with Unknown Objects

Hand pose estimation is important for human computer interaction, but the performance is not satisfying when the hand is interacting with objects. To alleviate the influence of unknown objects, we propose a novel weakly supervised segmentation guided scheme to estimate hand poses. Approximate hand m...

Full description

Saved in:
Bibliographic Details
Published inICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 2673 - 2677
Main Authors Zhang, Cairong, Wang, Guijin, Chen, Xinghao, Xie, Pengwei, Yamasaki, Toshihiko
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.05.2020
Subjects
Computer architecture
Convolution Neural Network
Feature extraction
Hand Object Interaction
Hand Pose Estimation
Human computer interaction
Image segmentation
Pose estimation
Speech processing
Task analysis
Weakly Supervision
Online AccessGet full text

Cover

More Information
Summary:Hand pose estimation is important for human computer interaction, but the performance is not satisfying when the hand is interacting with objects. To alleviate the influence of unknown objects, we propose a novel weakly supervised segmentation guided scheme to estimate hand poses. Approximate hand masks generated from annotations of sparse hand joints are used to supervise the segmentation task. Better features can be extracted since they are shared between the two tasks of hand segmentation and hand pose estimation. With the guidance of weakly supervised segmentation, the network can learn intermediate features balanced between focusing on the foreground and preserving contextual information. Finally the xy and z coordinates are estimated in different branches but utilizing shared feature maps. Experimental results of three different tasks on the publicly available FHAD dataset demonstrate the effectiveness of the proposed architecture.
ISSN:2379-190X
DOI:10.1109/ICASSP40776.2020.9053082