Deep Convolutional Neural Networks for Efficient Pose Estimation in Gesture Videos

• Published in: Computer Vision -- ACCV 2014 Vol. 9003; pp. 538 - 552
• Main Authors: Pfister, Tomas, Simonyan, Karen, Charles, James, Zisserman, Andrew
• Format: Book Chapter
• Language: English
• Published: Switzerland Springer International Publishing AG 2015; Springer International Publishing
• Series: Lecture Notes in Computer Science
• Subjects: Artificial intelligence; British Sign Language; Convolutional Layer; Convolutional Neural Network; Ground Truth; Image processing; Input Frame; Pattern recognition
• ISBN: 3319168649; 9783319168647
• ISSN: 0302-9743; 1611-3349
• DOI: 10.1007/978-3-319-16865-4_35

Our objective is to efficiently and accurately estimate the upper body pose of humans in gesture videos. To this end, we build on the recent successful applications of deep convolutional neural networks (ConvNets). Our novelties are: (i) our method is the first to our knowledge to use ConvNets for estimating human pose in videos; (ii) a new network that exploits temporal information from multiple frames, leading to better performance; (iii) showing that pre-segmenting the foreground of the video improves performance; and (iv) demonstrating that even without foreground segmentations, the network learns to abstract away from the background and can estimate the pose even in the presence of a complex, varying background. We evaluate our method on the BBC TV Signing dataset and show that our pose predictions are significantly better, and an order of magnitude faster to compute, than the state of the art [3].