Bayesian Integration of Face and Low-Level Cues for Foveated Video Coding

• Published in: IEEE transactions on circuits and systems for video technology Vol. 18; no. 12; pp. 1727 - 1740
• Main Authors: Boccignone, G., Marcelli, A., Napoletano, P., Di Fiore, G., Iacovoni, G., Morsa, S.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.12.2008; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Applied sciences; Bayesian methods; Coding, codes; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Face; Face detection; foveated video coding; foveation filtering; Humans; Image coding; Image processing; Information, signal and communications theory; Layout; Level control; Pattern recognition; Performance gain; Protocols; Signal and communications theory; Signal processing; Signal, noise; Telecommunications and information theory; Video coding; Video compression; video quality measurement; Biometrics; Video coding; Performance evaluation; Foveation; Image processing; video quality measurement; Data compression; foveated video coding; foveation filtering; Video signal processing; Image coding; Image evaluation; Prior information; Filtering; Face recognition; Image compression; Pattern recognition; Experimental protocol; Face detection; Image quality; Calling line identification presentation; Quality control; Miniaturization; Bayes methods; Automatic recognition; Gain
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2008.2005798

We present a Bayesian model that allows to automatically generate flxations/foveations and that can be suitably exploited for compression purposes. The twofold aim of this work is to investigate how the exploitation of high-level perceptual cues provided by human faces occurring in the video can enhance the compression process without reducing the perceived quality of the video and to validate such assumption with an extensive and principled experimental protocol. To such end, the model integrates top-down and bottom-up cues to choose the fixation point on a video frame: at the highest level, a fixation is driven by prior information and by relevant objects, namely human faces, within the scene; at the same time, local saliency together with novel and abrupt visual events contribute by triggering lower level control. The performance of the resulting video compression system has been evaluated with respect to both the perceived quality of foveated video clips and the compression gain with an extensive evaluation campaign, which has eventually involved 200 subjects.