Network-Based H.264/AVC Whole-Frame Loss Visibility Model and Frame Dropping Methods

• Published in: IEEE transactions on image processing Vol. 21; no. 8; pp. 3353 - 3363
• Main Authors: Yueh-Lun Chang, Ting-Lan Lin, Cosman, P. C.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.08.2012; Institute of Electrical and Electronics Engineers
• Subjects: Algorithms; Applied sciences; Coding, codes; Computer Communication Networks; Data Compression - methods; Decoding; Detection, estimation, filtering, equalization, prediction; Exact sciences and technology; Humans; Image Enhancement - methods; Image processing; Imaging, Three-Dimensional - methods; Information, signal and communications theory; Interpolation; Observers; Packet dropping policy; packet loss; Pattern Recognition, Automated - methods; perceptual video quality; Predictive models; Reproducibility of Results; Sample Size; Sensitivity and Specificity; Signal and communications theory; Signal processing; Signal Processing, Computer-Assisted; Signal, noise; Telecommunications and information theory; Video Recording - methods; Videos; visibility model; Visualization; Video coding; Packet switching; Image processing; visibility model; Data compression; Data transmission; Teletraffic; packet loss; Router; Image compression; Packet dropping policy; Decoding; Nodes; Video signal processing; perceptual video quality; Image quality; Interpolation; Image coding; Spurious signal; Observer; Transmission loss; Traffic congestion
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2012.2191567

We examine the visual effect of whole-frame loss by different decoders. Whole-frame losses are introduced in H.264/AVC compressed videos which are then decoded by two different decoders with different common concealment effects: frame copy and frame interpolation. The videos are seen by human observers who respond to each glitch they spot. We found that about 39% of whole-frame losses of B frames are not observed by any of the subjects, and over 58% of the B frame losses are observed by 20% or fewer of the subjects. Using simple predictive features that can be calculated inside a network node with no access to the original video and no pixel level reconstruction of the frame, we develop models that can predict the visibility of whole B frame losses. The models are then used in a router to predict the visual impact of a frame loss and perform intelligent frame dropping to relieve network congestion. Dropping frames based on their visual scores proves superior to random dropping of B frames.