Using iterative 3D-model fitting for domain adaptation of a hand-pose-estimation neural network

• Main Author: Lyons, Samuel John Llewellyn
• Format: Patent
• Language: English
• Published: 12.12.2023
• Subjects: CALCULATING; COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

Described is a solution for an unlabeled target domain dataset challenge using a domain adaptation technique to train a neural network using an iterative 3D model fitting algorithm to generate refined target domain labels. The neural network supports the convergence of the 3D model fitting algorithm and the 3D model fitting algorithm provides refined labels that are used for training of the neural network. During real-time inference, only the trained neural network is required. A convolutional neural network (CNN) is trained using labeled synthetic frames (source domain) with unlabeled real depth frames (target domain). The CNN initializes an offline iterative 3D model fitting algorithm capable of accurately labeling the hand pose in real depth frames. The labeled real depth frames are used to continue training the CNN thereby improving accuracy beyond that achievable by using only unlabeled real depth frames for domain adaptation.