EM-POSE: 3D Human Pose Estimation from Sparse Electromagnetic Trackers

• Published in: Proceedings / IEEE International Conference on Computer Vision pp. 11490 - 11500
• Main Authors: Kaufmann, Manuel, Zhao, Yi, Tang, Chengcheng, Tao, Lingling, Twigg, Christopher, Song, Jie, Wang, Robert, Hilliges, Otmar
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2021
• Subjects: 3D from multiview and other sensors; Gestures and body pose; Motion and tracking; Sensor systems; Shape; Shape measurement; Stereo; Three-dimensional displays; Tracking; Wireless communication; Wireless sensor networks
• DOI: 10.1109/ICCV48922.2021.01131

Fully immersive experiences in AR/VR depend on re-constructing the full body pose of the user without restricting their motion. In this paper we study the use of body-worn electromagnetic (EM) field-based sensing for the task of 3D human pose reconstruction. To this end, we present a method to estimate SMPL parameters from 6-12 EM sensors. We leverage a customized wearable system consisting of wireless EM sensors measuring time-synchronized 6D poses at 120 Hz. To provide accurate poses even with little user instrumentation, we adopt a recently proposed hybrid framework, learned gradient descent (LGD), to iteratively estimate SMPL pose and shape from our input measurements. This allows us to harness powerful pose priors to cope with the idiosyncrasies of the input data and achieve accurate pose estimates. The proposed method uses AMASS to synthesize virtual EM-sensor data and we show that it generalizes well to a newly captured real dataset consisting of a total of 36 minutes of motion from 5 subjects. We achieve reconstruction errors as low as 31.8 mm and 13.3 degrees, outperforming both pure learning- and pure optimization-based methods. Code and data is available under https://ait.ethz.ch/projects/2021/em-pose.