Multisensory Experience with Images

• Published in: MAMNA : 2014 Spring Symposium : From Lab to Life: Field Based Applications of MEMS & NEMS : March 10, 2014, Baltimore MD pp. 55 - 62
• Main Authors: Kan Chen, Xingzi Zhang, Rasool, Shahzad, Johan, Henry, Sourin, Alexei
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.10.2014
• Subjects: Computational modeling; Force; Geometry; haptic interaction; Haptic interfaces; Harmonic analysis; image warping; immersion; Rendering (computer graphics); tangible images; Three-dimensional displays
• DOI: 10.1109/CW.2014.16

We augment images with additional data allowing us to visualize them with elements of animation as well as interact with the images using haptic devices as if they were actual 3D scenes. First, we apply procedural simulation to simulate image distortions, which are characteristic for various phenomena like blowing wind, fluid flow, etc. Next, we consider how haptic rendering of images can be performed. This in turn includes haptic rendering of forces augmented to the images as well as haptic interaction with haptic objects augmented to the images. The immersive images thus obtained can be exchanged across the internet to realistically simulate 3D virtual scenes while in fact still be 2D images augmented with rather little additional data files in a way how movie subtitles files are added to the video files. The proposed approach is illustrated with a few representative examples.