Integration of Web of Tactile Things for Soft Vision-Based Tactile Sensor Toward Immersive Human-Robot Interaction

• Published in: 2024 IEEE/SICE International Symposium on System Integration (SII) pp. 1343 - 1348
• Main Authors: Dinh Le, Nhat Minh, Nguyen, Tuan Tai, Luu, Quan Khanh, Nguyen, Nhan Huu, Pham, Van Cu, Tan, Yasuo, Ho, Van Anh
• Format: Conference Proceeding
• Language: English
• Published: IEEE 08.01.2024
• Subjects: Artificial neural networks; Deformation; Sensors; Servers; Tactile sensors; Task analysis; Wireless communication
• ISSN: 2474-2325
• DOI: 10.1109/SII58957.2024.10417344

Web of Tactile Things (WoTT) aims toward a standardized platform designed for tactile devices, facilitating the exchange and updating of data across diverse domain web services. However, the requirements for specified devices pose a challenge when deploying on low-profile devices. Furthermore, the concentration of processing the sensing and server operations on a single computer creates a bottleneck that restricts the system's scalability. To enhance the potential applications, this study introduces an extended platform based on WoTT that decentralizes processing. This separation involves distinct processing for the tactile sensor and the WoTT server becomes independent elements. The tactile sensor is integrated with an onboard computer for sensing tasks, enabling wireless transmission of tactile data packets. In this proposed system, each external device can establish wireless connections with the server via the MQTT (Message Queueing Telemetry Transport) message protocol. In order to validate this approach, we construct a comprehensive system incorporating tactile sensors, AR devices, and 2D linear stages. Leveraging a deep neural network (DNN), the tactile sensor's data drives the anticipation of soft skin deformation by capturing visual cues through marker displacement. Tactile data is then encoded and transmitted to end devices using the MQTT protocol, directing the 2D linear stage's interaction with a phantom arm, while AR devices visualize the tactile sensor's soft skin deformation.