Single Shot Corrective CNN for Anatomically Correct 3D Hand Pose Estimation

• Published in: Frontiers in artificial intelligence Vol. 5; p. 759255
• Main Authors: Isaac, Joseph H R, Manivannan, Muniyandi, Ravindran, Balaraman
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 21.02.2022
• Subjects: 3D hand pose estimation; anatomically correct tracking; Artificial Intelligence; biomechanical constraints; depth based hand tracking; single shot corrective CNN; anatomically correct tracking; depth based hand tracking; biomechanical constraints; single shot corrective CNN; 3D hand pose estimation
• ISSN: 2624-8212; 2624-8212
• DOI: 10.3389/frai.2022.759255

Hand pose estimation in 3D from depth images is a highly complex task. Current state-of-the-art 3D hand pose estimators focus only on the accuracy of the model as measured by how closely it matches the ground truth hand pose but overlook the resulting hand pose's anatomical correctness. In this paper, we present the Single Shot Corrective CNN (SSC-CNN) to tackle the problem of enforcing anatomical correctness at the architecture level. In contrast to previous works which use post-facto pose filters, SSC-CNN predicts the hand pose that conforms to the human hand's biomechanical bounds and rules in a single forward pass. The model was trained and tested on the HANDS2017 and MSRA datasets. Experiments show that our proposed model shows comparable accuracy to the state-of-the-art models as measured by the ground truth pose. However, the previous methods have high anatomical errors, whereas our model is free from such errors. Experiments show that our proposed model shows zero anatomical errors along with comparable accuracy to the state-of-the-art models as measured by the ground truth pose. The previous methods have high anatomical errors, whereas our model is free from such errors. Surprisingly even the ground truth provided in the existing datasets suffers from anatomical errors, and therefore Anatomical Error Free (AEF) versions of the datasets, namely AEF-HANDS2017 and AEF-MSRA, were created.