Semantic annotation of narrative media objects

• Published in: Multimedia tools and applications Vol. 59; no. 2; pp. 407 - 439
• Main Authors: Lombardo, Vincenzo, Damiano, Rossana
• Format: Journal Article
• Language: English
• Published: Boston Springer US 01.07.2012; Springer Nature B.V
• Subjects: Annotations; Archives & records; Composers; Computer Communication Networks; Computer programs; Computer Science; Cultural heritage; Data Structures and Information Theory; Design engineering; Editing; Media; Metadata; Motion pictures; Multimedia; Multimedia computer applications; Multimedia Information Systems; Narratives; Ontology; Representations; Semantics; Software; Special Purpose and Application-Based Systems; Writers; Automatic video editing; Multimedia corpus annotation; Narrative media object
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-011-0813-2

This paper addresses the annotation of the narrative features of media objects. Based on a relevant narratological and computational background, we introduce an ontology–based model called Drammar, an annotation schema for the narrative features of media objects based on Drammar and a software tool, Cinematic, for annotating these objects and validating the annotation. Annotated media objects can also be automatically edited into sequences, with the twofold goal of testing the validity of the annotation—through the reconstruction of the baseline sequence—and exploring the possibility of alternative sequences. The software tool encodes both the narrative model and the annotation itself in ontological format, and relies on external ontologies for representing world knowledge and limit the arbitrariness of the annotation. The paper opens the way to the design of a general annotation schema for narrative multimedia with the long–term goal of building large corpora of annotated video material and of bridging the gap between the low–level signal analysis and the high–level semantic representation of the narrative content of the media objects. Finally, the paper illustrates a few projects elaborated with the Drammar annotation and the Cinematic tool, with purposes of artistic research and cross–media analysis, that provide an empirical validation of the annotation process.