Efficient mode decision scheme based on edge detection with Gaussian pulse for Intra-prediction in H.264/AVC

• Published in: Alexandria engineering journal Vol. 61; no. 4; pp. 2709 - 2722
• Main Authors: El-Mowafy, M.A., Gharghory, S.M., Abo-Elsoud, M.A., Obayya, M., Fath Allah, M.I.
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 01.04.2022; Elsevier
• Subjects: Edge detection-based gradient; Gaussian pulse; H.264/AVC Video Coding; Homogeneity; Intra-prediction; Mode decision; Homogeneity; H.264/AVC Video Coding; Gaussian pulse; Intra-prediction; Mode decision; Edge detection-based gradient
• ISSN: 1110-0168
• DOI: 10.1016/j.aej.2021.07.044

Advanced Video Coding (H.264/AVC) is one of the most important video coding standards which was developed by the ITU-T Video Coding Experts Group (VCEG) together with the ISO/IEC Moving Picture Experts Group (MPEG). Due to the complex procedures needed to find the optimal mode in intra prediction mode stage of H.264/AVC, an efficient mode decision is suggested in this paper. The main objectives of the proposed work are improving the compression efficiency of video coding and minimizing the degradation of video quality. The proposed algorithm is based on the gradient method for edge detection using the Prewitt filter to determine the texture direction of a block to be predicted in the luminance component of a video frame. To decide either implemented the DC mode or the edge-detection based gradient method to predict block under consideration, a predefined threshold value is used for the homogeneity test of each macro-block. To reduce the Bit Rate of the compressed video, Gaussian pulse modulation is also suggested to the Discrete Cosine Transformation (DCT) components of each macro-block during the coding process. The proposed algorithm is carried out on a luminance component of a set of different YUV video sequences with different resolutions and different quantization parameters using MATLAB. The suggested algorithm is evaluated using different metrics which are Peak Signal to Noise Ratio (PSNR) and bit rate measured by Bjontegaard Delta method. The simulation results show the ability of the proposed algorithm in enhancing the BDPSNR averagely by 1.14 dB and reducing the BDBR averagely by 11.34 %for QCIF sequences compared to the existing state-of-the-art fast intra-prediction mode decision algorithms.