Laplace Distribution Based Lagrangian Rate Distortion Optimization for Hybrid Video Coding

• Published in: IEEE transactions on circuits and systems for video technology Vol. 19; no. 2; pp. 193 - 205
• Main Authors: Xiang Li, Oertel, N., Hutter, A., Kaup, A.
• Format: Journal Article
• Language: English
• Published: New York, NY IEEE 01.02.2009; Institute of Electrical and Electronics Engineers; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Applied sciences; Automatic voltage control; Circuits; Coding; Distortion; Exact sciences and technology; Gain; Hybrid video coding; Image processing; Information, signal and communications theory; Lagrange multiplier selection; Lagrangian functions; Mathematical models; Multimedia communication; Noise levels; Optimization; Optimization methods; Quantization; Rate-distortion; rate-distortion optimization (RDO); Sampling, quantization; Signal and communications theory; Signal processing; Software algorithms; Telecommunications and information theory; Video coding; Video coding; Performance evaluation; Rate distortion theory; Quantum effect; Algorithm; Adaptive method; Optimization; Video signal processing; Lagrange multiplier; Hybrid video coding; Simulation; Signal quantization; Software; Hybrid model; Laplace transformation; Lagrange multiplier selection; rate-distortion optimization (RDO); Gain; Signal to noise ratio
• ISSN: 1051-8215; 1558-2205
• DOI: 10.1109/TCSVT.2008.2009255

In today's hybrid video coding, Rate-Distortion Optimization (RDO) plays a critical role. It aims at minimizing the distortion under a constraint on the rate. Currently, the most popular RDO algorithm for one-pass coding is the one recommended in the H.264/AVC reference software. It, or HR-lambda for convenience, is actually a kind of universal method which performs the optimization only according to the quantization process while ignoring the properties of input sequences. Intuitively, it is not efficient all the time and an adaptive scheme should be better. Therefore, a new algorithm Lap- lambda is presented in this paper. Based on the Laplace distribution of transformed residuals, the proposed Lap-lambda is able to adaptively optimize the input sequences so that the overall coding efficiency is improved. Cases which cannot be well captured by the proposed models are considered via escape methods. Comprehensive simulations verify that compared with HR-lambda , Lap-lambda shows a much better or similar performance in all scenarios. Particularly, significant gains of 1.79 dB and 1.60 dB in PSNR are obtained for slow sequences and B-frames, respectively.