Learning for Video Compression

• Published in: IEEE transactions on circuits and systems for video technology Vol. 30; no. 2; pp. 566 - 576
• Main Authors: Chen, Zhibo, He, Tianyu, Jin, Xin, Wu, Feng
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.02.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Codec; Codecs; Coding; Image coding; Image reconstruction; Iterative methods; Learning; Neural networks; Performance prediction; PixelMotionCNN; Spatiotemporal phenomena; Transform coding; Video coding; Video compression
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2019.2892608

One key challenge to learning-based video compression is that motion predictive coding, a very effective tool for video compression, can hardly be trained into a neural network. In this paper, we propose the concept of Pixel-MotionCNN (PMCNN) which includes motion extension and hybrid prediction networks. PMCNN can model spatiotemporal coherence to effectively perform predictive coding inside the learning network. On the basis of PMCNN, we further explore a learning-based framework for video compression with additional components of iterative analysis/synthesis and binarization. The experimental results demonstrate the effectiveness of the proposed scheme. Although entropy coding and complex configurations are not employed in this paper, we still demonstrate superior performance compared with MPEG-2 and achieve comparable results with H.264 codec. The proposed learning-based scheme provides a possible new direction to further improve compression efficiency and functionalities of future video coding.