Man-machine interaction virtual-real fusion method and device based on mixed reality head-mounted display

• Main Authors: YANG ZHIGANG, XU SHUHONG, CHENG TIANZHUANG, YANG DONGHAO, WU CHENGCHENG, WANG DAWEI
• Format: Patent
• Language: Chinese; English
• Published: 09.08.2022
• Subjects: CALCULATING; COMPUTING; COUNTING; ELECTRIC DIGITAL DATA PROCESSING; IMAGE DATA PROCESSING OR GENERATION, IN GENERAL; PHYSICS

The invention discloses a human-computer interaction virtual-real fusion method and device based on a mixed reality head-mounted display. The method comprises the following steps: acquiring an internal parameter matrix M < internal > and an external parameter matrix M < external > of a camera; and according to the internal parameter matrix and the external parameter matrix, obtaining tracking coordinate system related information {Pori, Roi} of the MR equipment and a pose relationship between the entity camera and a tracking origin, obtaining an external parameter M'of the camera, and according to the internal parameter and the external parameter of the camera, obtaining a fused image. The method solves the problem that in the prior art, in order to achieve interactivity under the two conditions, a virtual human model of which the clothes, the skin color, the skeleton length and the like are similar to those of a testee is generally constructed in a virtual scene by means of a modeling technology, and the vir