STIMONT: a core ontology for multimedia stimuli description

• Published in: Multimedia tools and applications Vol. 73; no. 3; pp. 1103 - 1127
• Main Authors: Horvat, Marko, Bogunović, Nikola, Ćosić, Krešimir
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.12.2014; Springer; Springer Nature B.V
• Subjects: Analysis; Applied sciences; Artificial intelligence; Computer Communication Networks; Computer programs; Computer Science; Computer science; control theory; systems; Computer systems and distributed systems. User interface; Data processing. List processing. Character string processing; Data Structures and Information Theory; Documents; Emotions; Exact sciences and technology; Information systems. Data bases; Keywords; Memory organisation. Data processing; Metadata; Multimedia; Multimedia computer applications; Multimedia Information Systems; Ontology; Pattern recognition. Digital image processing. Computational geometry; Physiology; Recall; Retrieval; Semantics; Software; Special Purpose and Application-Based Systems; Stimuli; Studies; Multimedia; Stimulus; Emotion; Ontology; OWL; Database query; Keyword; Psychology; Web service; Video signal; Multimedia databases; Metadata; Information retrieval; Emotion emotionality; Document retrieval; Semantic web; Software tool; Neuroscience; Document layout; Semantics; Physiology; Recall
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-013-1624-4

Affective multimedia documents such as images, sounds or videos elicit emotional responses in exposed human subjects. These stimuli are stored in affective multimedia databases and successfully used for a wide variety of research in psychology and neuroscience in areas related to attention and emotion processing. Although important all affective multimedia databases have numerous deficiencies which impair their applicability. These problems, which are brought forward in the paper, result in low recall and precision of multimedia stimuli retrieval which makes creating emotion elicitation procedures difficult and labor-intensive. To address these issues a new core ontology STIMONT is introduced. The STIMONT is written in OWL-DL formalism and extends W3C EmotionML format with an expressive and formal representation of affective concepts, high-level semantics, stimuli document metadata and the elicited physiology. The advantages of ontology in description of affective multimedia stimuli are demonstrated in a document retrieval experiment and compared against contemporary keyword-based querying methods. Also, a software tool Intelligent Stimulus Generator for retrieval of affective multimedia and construction of stimuli sequences is presented.