Calibration of inertial measurement units in alignment with a skeleton model to control a computer system based on determination of orientation of an inertial measurement unit from an image of a portion of a user

• Main Authors: Gusev, Alexey Andreevich, Kartashov, Alexey Ivanovich, Erivantcev, Viktor Vladimirovich, Gubaidullin, Ratmir Rasilevich, Goncharov, Daniil Olegovich
• Format: Patent
• Language: English
• Published: 13.04.2021
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

A method to calibrate orientation measurements of an inertial measurement unit of a sensor device based on an image of a portion of a user to which the sensor device is attached. For example, the sensor device can be configured to be attached to the middle phalange of the index finger and configured with a touch pad. In response to the determination that the thumb of the user is placed on the touch pad of the sensor device, the camera of the system can capture the image showing that the hand of the user. A convolutional neural network is configured to determine, from the image, orientations of predefined features of the hand of the user. A further artificial neural network is configured to determine the orientation of the sensor device based on the orientations of the predefined features to calibrate the orientation measurements of the inertial measurement unit.