Architecture for Embedding Audiovisual Feature Extraction Tools in Archives

• Published in: SMPTE Annual Tech Conference & Expo, 2009 pp. 1 - 10
• Main Authors: De Sutter, Robbie, Braeckman, Karel
• Format: Conference Proceeding
• Language: English
• Published: SMPTE 01.10.2009
• Subjects: Annotation; Architecture; Archives; Feature Extraction Tools; File-based Production; Integrated Workflow; Media Asset Management
• ISBN: 9781614829430; 1614829438
• DOI: 10.5594/M001332

In the near future, it will no longer be sufficient that only archivists annotate audiovisual material. Not only is the number of archivists limited, the time they can spend on annotating one item is insufficient to create time-based and detailed descriptions about the content to make fully optimized video search possible. Furthermore, we observe an accelerated increase in newly created audiovisual material that must be described due to introduction of file-based production methods. - Fortunately, more and more high-quality feature extraction tools are being developed by research institutes. These tools examine the audiovisual essence and return particular information about the analyzed video and/or audio streams. For example, tools can automatically detect shot boundaries, detect and recognize faces and objects, segment audio streams, etc. As a result, they quickly and cheaply generate metadata that can be used for indexing and searching. On top of that, it relieves archivists of performing tedious and repetitive, but necessary low-added value tasks, for example identifying within an audio stream the speech and music segments. Although most tools are currently not yet commercially offered, it is to be expected that these solutions soon will become available for broadcasters and media companies alike. - In this paper, we describe a solution on how to integrate such feature extraction tools within the annotation workflow of a media company. This solution, in the form of an architecture and workflow, is scalable, extensible, loosely coupled, and has clear and easy to implement interfaces. As such, our architecture allows one to plug in additional tools irrespective of the software and hardware used by the media company. - By integrating feature extraction tools within the workflow of annotating audiovisual essence, more and better metadata can be created allowing other tools to improve indexing, search and retrieval of media material within audiovisual archives.