Attention-based video streaming

• Published in: Signal processing. Image communication Vol. 25; no. 10; pp. 745 - 760
• Main Authors: DIKICI, Cağatay, BOZMA, H. Işil
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier B.V 01.11.2010; Elsevier
• Subjects: Algorithms; Applied sciences; Artificial intelligence; Biological; Biologically motivated attentive vision; Computer science; control theory; systems; Connectionism. Neural networks; Degradation; Exact sciences and technology; Face tracking; Foveation; Frames; Image processing; Information, signal and communications theory; Networks; Neural networks; Partitions; Pattern recognition; Pattern recognition. Digital image processing. Computational geometry; Servers; Signal processing; Spatio-temporal pre-processing; Telecommunications and information theory; Video streaming; Spatio-temporal pre-processing; Biologically motivated attentive vision; Foveation; Video streaming; Face tracking; Neural networks; Biometrics; High resolution; Video signal; Vision system; Video signal processing; Learning; Degradation; Backpropagation algorithm; Image sequence; Streaming; Computer vision; Face recognition; Pattern recognition; Neural network; Information transmission; Temporal logic; Time resolution; A priori estimation; Automatic recognition; Spatial resolution
• ISSN: 0923-5965; 1879-2677
• DOI: 10.1016/j.image.2010.08.002

This paper considers the problem of video streaming in low bandwidth networks and presents a complete framework that is inspired by the fovea–periphery distinction of biological vision systems. First, an application specific attention function that serves to find the important small regions in a given frame is constructed a priori using a back-propagation neural network that is optimized combinatorially. Given a specific application, the respective attention function partitions each frame into foveal and periphery regions and then a spatial–temporal pre-processing algorithm encodes the foveal regions with high spatial resolution while the periphery regions are encoded with lower spatial and temporal resolution. Finally, the pre-processed video sequence is streamed using a standard streaming server. As an application, we consider the transmission of human face videos. Our experimental results indicate that even with limited amount of training, the constructed attention function is able to determine the foveal regions which have improved transmission quality while the peripheral regions have an acceptable degradation.