TexSenseGAN: A User-Guided System for Optimizing Texture-Related Vibrotactile Feedback Using Generative Adversarial Network

• Published in: IEEE transactions on haptics Vol. 18; no. 2; pp. 325 - 339
• Main Authors: Zhang, Mingxin, Terui, Shun, Makino, Yasutoshi, Shinoda, Hiroyuki
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.04.2025
• Subjects: Aerospace electronics; autoencoder; Deep learning; Feedback, Sensory - physiology; Generative Adversarial Networks; Generators; Haptic display; Haptic interfaces; Human in the loop; human-computer interaction; Humans; Neural Networks, Computer; Optimization; Touch; Touch Perception - physiology; Training; User-Computer Interface; Vectors; Vibration; Vibrations
• ISSN: 1939-1412; 2329-4051; 2329-4051
• DOI: 10.1109/TOH.2025.3542424

Vibration rendering is essential for creating realistic tactile experiences in human-virtual object interactions, such as in video game controllers and VR devices. By dynamically adjusting vibration parameters based on user actions, these systems can convey spatial features and contribute to texture representation. However, generating arbitrary vibrations to replicate real-world material textures is challenging due to the large parameter space. This study proposes a human-in-the-loop vibration generation model based on user preferences. To enable users to easily control the generation of vibration samples with large parameter spaces, we introduced an optimization model based on Differential Subspace Search (DSS) and Generative Adversarial Network (GAN). With DSS, users can employ a one-dimensional slider to easily modify the high-dimensional latent space to ensure that the GAN can generate desired vibrations. We trained the generative model using an open dataset of tactile vibration data and selected five types of vibrations as target samples for the generation experiment. Extensive user experiments were conducted using the generated and real samples. The results indicated that our system could generate distinguishable samples that matched the target characteristics. Moreover, we established a correlation between subjects' ability to distinguish real samples and their ability to distinguish generated samples.