COMPACT HAPTIC MIXED REALITY SIMULATOR

• Main Authors: LEIBACHER, Ivo, TUCHSCHMID, Stefan, OELHAFEN, Markus, MATTES-O'BRIEN, Carolyn, BACHOFEN, Daniel, HENSCHEL, Christopher
• Format: Patent
• Language: English; French; German
• Published: 24.07.2024
• Subjects: ADVERTISING; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND,DEAF OR MUTE; CRYPTOGRAPHY; DIAGRAMS; DISPLAY; EDUCATION; EDUCATIONAL OR DEMONSTRATION APPLIANCES; GLOBES; PHYSICS; PLANETARIA; SEALS

A compact and versatile mixed reality simulator system and methods may comprise a mixed reality processing engine and a haptic control engine to control a linear electromagnetic motor (LEM) of 1 degree of freedom made of a handheld instrument replicate and a trocar-like instrument hosting duct. The compact and versatile mixed reality simulator system may comprise multiple interchangeable instrument hosting ducts and handheld instrument replicates and the mixed reality processing engine may automatically detect, from sensors, which instrument replicate has been inserted in which duct. The mixed reality processing engine may detect a virtual contact between the handheld instrument replicate and a virtual object in a mixed reality scenario. The mixed reality processing engine may calculate with a real-time solver the kinesthetic force feedback signal to power the LEM as a function of the position and orientation of the instrument tracked from sensors, and of the position, orientation and material property of the virtual object as it deforms due to the virtual contact according to the mixed reality scenario. The mixed reality processing engine may adapt in real-time the haptic feedback by rescaling its magnitude, and/or combining it with a vibration signal, possibly with an additional vibrotactile actuator arrangement. The mixed reality processing engine may jointly adapt the haptic feedback signal and the virtual reality scene rendering for a more realistic mixed reality experience with a haptic retargeting method such as space warping.