SAPTE: A multimedia information system to support the discourse analysis and information retrieval of television programs

• Published in: Multimedia tools and applications Vol. 74; no. 23; pp. 10923 - 10963
• Main Authors: Pereira, Moisés H. R., de Souza, Celso L., Pádua, Flávio L. C., Silva, Giani D., de Assis, Guilherme T., Pereira, Adriano C. M.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.12.2015; Springer Nature B.V
• Subjects: African American studies; Algorithms; Analysis; Archives & records; Computer Communication Networks; Computer programs; Computer Science; Data Structures and Information Theory; Diachronic analysis; Digital video; Dimensional analysis; Discourse analysis; Image retrieval; Indexing; Information retrieval; Information systems; Linguistics; Metadata; Multimedia; Multimedia computer applications; Multimedia Information Systems; Research centers; Researchers; Software; Special Purpose and Application-Based Systems; Television; Television programs; Texture; User generated content; Content-based video retrieval; Discourse analysis; Television; Video indexing
• DOI: 10.1007/s11042-014-2311-9
• ISSN: 1380-7501

This paper presents a novel multimedia information system, called SAPTE, for supporting the discourse analysis and information retrieval of television programs from their corresponding video recordings. Unlike most common systems, SAPTE uses both content independent and dependent metadata, which are determined by the application of discourse analysis techniques as well as image and audio analysis methods. The proposed system was developed in partnership with the free-to-air Brazilian TV channel Rede Minas in an attempt to provide TV researchers with computational tools to assist their studies about this media universe. The system is based on the Matterhorn framework for managing video libraries, combining: (1) discourse analysis techniques for describing and indexing the videos, by considering aspects, such as, definitions of the subject of analysis, the nature of the speaker and the corpus of data resulting from the discourse; (2) a state of the art decoder software for large vocabulary continuous speech recognition, called Julius; (3) image and frequency domain techniques to compute visual signatures for the video recordings, containing color, shape and texture information; and (4) hashing and k -d tree methods for data indexing. The capabilities of SAPTE were successfully validated, as demonstrated by our experimental results, indicating that SAPTE is a promising computational tool for TV researchers.