Who Takes What: Using RGB-D Camera and Inertial Sensor for Unmanned Monitor

Advanced Internet of Things (IoT) techniques have made human-environment interaction much easier. Existing solutions usually enable such interactions without knowing the identities of action performers. However, identifying users who are interacting with environments is a key to enable personalized...

Full description

Saved in:
Bibliographic Details
Published in2019 International Conference on Robotics and Automation (ICRA) pp. 8063 - 8069
Main Authors Kao, Hsin-Wei, Ke, Ting-Yuan, Lin, Kate Ching-Ju, Tseng, Yu-Chee
Format Conference Proceeding
LanguageEnglish
Published IEEE 01.05.2019
Subjects
Cameras
Feature extraction
Monitoring
Reliability
Robot sensing systems
Skeleton
Trajectory
Online AccessGet full text

Cover

More Information
Summary:Advanced Internet of Things (IoT) techniques have made human-environment interaction much easier. Existing solutions usually enable such interactions without knowing the identities of action performers. However, identifying users who are interacting with environments is a key to enable personalized service. To provide such add-on service, we propose WTW (who takes what), a system that identifies which user takes what object. Unlike traditional vision-based approaches, which are typically vulnerable to blockage, our WTW combines the feature information of three types of data, i.e., images, skeletons and IMU data, to enable reliable user-object matching and identification. By correlating the moving trajectory of a user monitored by inertial sensors with the movement of an object recorded in the video, our WTW reliably identifies a user and matches him/her with the object on action. Our prototype evaluation shows that WTW achieves a recognition rate of over 90% even in a crowd. The system is reliable even when users locate close by and take objects roughly at the same time.
ISSN:2577-087X
DOI:10.1109/ICRA.2019.8793858