Real-time egocentric superimposition of operator's own body on telexistence avatar in virtual environment

• Published in: 2013 23rd International Conference on Artificial Reality and Telexistence (ICAT) pp. 35 - 39
• Main Authors: Saraiji, Mhd Yamen, Fernando, Charith Lasantha, Furukawa, Masahiro, Minarnizawa, Kouta, Tachi, Susumu
• Format: Conference Proceeding
• Language: English
• Published: VRSJ 01.12.2013
• Subjects: Cameras; Image color analysis; Kinematics; Lighting; Robots; Self-superimposing; Telexistence; Video See-through HMD; Virtual environments; Visualization
• DOI: 10.1109/ICAT.2013.6728903

During teleoperation manipulation, the synchronization of the user behavior and a remote avatar is important to deliver the sensation of being in that remote place. Current telexistence technologies allow full upper body posture synchronization through multi-DOF humanoid robot structures and allow the operator to control the remote body as his own. However, it does not preserve a consistent feedback, such as the human like skin tones, operator's hand shape and the current outfit he is wearing during the operation. Thus in this paper we propose a new method that provides the operator's body shape, complexion, and light correction using real-time visuals taken from a see-through camera placed in the HMD and superimposed over robot vision. By using hand and arm trajectory from a kinematics solver, a virtual representation is used to generate masking images that isolate his local body appearance and superimpose it into the virtual environment. Local body appearance is captured via a see-through HMD. This paper describes the design and implementation of the above technique and obtained basic results.