MODEL FOR PERSON PSYCHOEMOTIONAL STATE DETERMINATION USING AUDIO AND VIDEO DATA

• Published in: Research Result Information Technologies Vol. 7; no. 3
• Main Authors: Demin, O.D., Laptev, A.A.
• Format: Journal Article
• Language: English
• Published: 30.09.2022
• ISSN: 2518-1092; 2518-1092
• DOI: 10.18413/2518-1092-2022-7-3-0-5

Determination of the person’s psychoemotional state finds application in a variety of tasks from the diagnosis of diseases to the prevention of emergency traffic situations. Given the importance and variety of tasks where it used, there are many methods and approaches of it exists, but most of them use only one nonverbal feature, such as a person's voice or facial expressions. At the same time, the simultaneous use of several nonverbal signs can increase accuracy compared to methods using only one nonverbal sign. In this paper, such method has been developed that uses voice and facial expression as nonverbal signs. Based on the literature review and analysis of existing methods, the best methods and approaches using voice or facial expression were selected, the developed method was compared with existing solutions. The developed method made it possible to improve the accuracy of CMCNN, but at the same time, in each test, the Accuracy of its own method was less in comparison with DisVoice + SVM. For improving developed method weighted average was proposed to use instead of usual average or using more complex model, which is able to detect interrelation between person’s voice and facial expression.