Taxonomy of Directing Semantics for Film Shot Classification

• Published in: IEEE transactions on circuits and systems for video technology Vol. 19; no. 10; pp. 1529 - 1542
• Main Authors: HEE LIN WANG, CHEONG, Loong-Fah
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.10.2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Attention; Cameras; Cinematography; Classification; Computer vision; Construction; Exact sciences and technology; film directing semantics; film grammar motion; Image processing; Indexing; Information, signal and communications theory; Markov random fields; Miscellaneous; Motion pictures; Motion segmentation; movie shot; Robustness; Semantics; Shot; Signal processing; Studies; Taxonomy; Telecommunications and information theory; Videos; Performance evaluation; Probabilistic approach; Motion estimation; Image processing; Background; Semantic analysis; Video signal; Attention; Cinematography; film grammar motion; Mapping; Grammar; Markov model; classification; Algorithm; Random field; Image segmentation; movie shot; Signal processing; Foreground; film directing semantics; Indexing
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2009.2022705

The immense indexing potential of motion cues has hitherto been realized only in domains with more apparent structure (e.g., sport videos). To address the lack of theoretical attention and to realize the potential of motion-based indexing in the subtler film domain, we propose a systematic approach to build taxonomy for film directing semantics. These motion-related semantics are grounded upon cinematography and are thus more appealing to users. In order to automate the classification of these semantics, we have developed a novel Markov random field based motion segmentation algorithm with an integral foreground/background identification capability based on edge occlusion reasoning. This algorithm is sufficiently robust and fast for film domain conditions, and allows us to formulate salient and novel motion descriptors capable of mapping to the proposed directing semantics. We demonstrate the validity of the framework and effectiveness of the motion-based descriptors by classifying shots from Hollywood domain movies according to the proposed taxonomy with satisfactory results.