Baptizo: A sensor fusion based model for tracking the identity of human poses

• Published in: Information fusion Vol. 62; pp. 1 - 13
• Main Authors: Bazo, Rodrigo, Reis, Eduardo, Seewald, Lucas Adams, Rodrigues, Vinicius Facco, da Costa, Cristiano André, Gonzaga, Luiz, Antunes, Rodolfo Stoffel, Righi, Rodrigo da Rosa, Maier, Andreas, Eskofier, Björn, Fahrig, Rebecca, Horz, Tim
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.10.2020
• Subjects: Activity recognition; HPE; Radiofrequency tracking; RTLS; Sensor fusion; Vision tracking; Radiofrequency tracking; Sensor fusion; Activity recognition; Vision tracking; HPE; RTLS
• ISSN: 1566-2535; 1872-6305
• DOI: 10.1016/j.inffus.2020.03.011

•Human poses and radio id fusion can create valuable activity recognition datasets.•Vision and radio devices data fusion enable assessing each technology limitation.•Body joints estimated with ToF devices enable radio tracking accuracy improvement.•Radiofrequency tracking errors can be reduced up to 46% through data fusion. Recent advances in the capabilities of computing devices enable new methods to estimate the pose of humans. Human pose estimation techniques are relevant for several industry fields, such as surveillance and interactive entertainment. Further, encoded human poses provide a valuable input for behavioral analysis and activity recognition. Body part detectors offer millimetric accuracy thanks to state-of-the-art Computer Vision technology. However, they still suffer from issues, such as long-term occlusion, that hinder the identification of human subjects. Such problems are intrinsic to Computer Vision devices and can only be solved either with the use of heuristic methods or the deployment of more cameras, which are not always feasible. In turn, radiofrequency-based tracking systems do not suffer from occlusion or identity loss problems and, albeit not as precise as Computer Vision methods, can achieve a high accuracy level. Radiofrequency positioning systems and human pose estimation techniques can complement each other in different ways. For example, the prior can help to identify tracked humans and reduce occlusion errors, while the later can increase the accuracy of obtained positions. Thus, the combination of radiofrequency-based positioning and computer vision-based human pose estimation yields a solution that provides better tracking results. Therefore, this article proposes a system that generates identified pose data by fusing the unique identities of radiofrequency sensors with unidentified body poses while using estimated body parts for reducing radiofrequency position estimations errors. Experiments with a proof-of-concept demonstrate the feasibility of the sensor fusion technique and also show a potential reduction on positioning errors by nearly 46%.