A Method for Measuring Contact Points in Human–Object Interaction Utilizing Infrared Cameras

• Published in: Frontiers in robotics and AI Vol. 8; p. 800131
• Main Authors: Hakala, Jussi, Häkkinen, Jukka
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Media S.A 14.02.2022
• Subjects: contact point; grasping; infrared camera; prehension movements; Robotics and AI; touch; grasping; touch; infrared camera; contact point; prehension movements
• ISSN: 2296-9144; 2296-9144
• DOI: 10.3389/frobt.2021.800131

This article presents a novel method for measuring contact points in human–object interaction. Research in multiple prehension-related fields, e.g., action planning, affordance, motor function, ergonomics, and robotic grasping, benefits from accurate and precise measurements of contact points between a subject’s hands and objects. During interaction, the subject’s hands occlude the contact points, which poses a major challenge for direct optical measurement methods. Our method solves the occlusion problem by exploiting thermal energy transfer from the subject’s hand to the object surface during interaction. After the interaction, we measure the heat emitted by the object surface with four high-resolution infrared cameras surrounding the object. A computer-vision algorithm detects the areas in the infrared images where the subject’s fingers have touched the object. A structured light 3D scanner produces a point cloud of the scene, which enables the localization of the object in relation to the infrared cameras. We then use the localization result to project the detected contact points from the infrared camera images to the surface of the 3D model of the object. Data collection with this method is fast, unobtrusive, contactless, markerless, and automated. The method enables accurate measurement of contact points in non-trivially complex objects. Furthermore, the method is extendable to measuring surface contact areas, or patches, instead of contact points. In this article, we present the method and sample grasp measurement results with publicly available objects.