Long-Range Dependencies and High-Order Spatial Pooling for Deep Model-Based Full-Reference Image Quality Assessment

• Published in: IEEE access Vol. 8; pp. 72007 - 72020
• Main Authors: Liu, Mengyang, Po, Lai-Man, Xu, Xuyuan, Cheung, Kwok Wai, Zhao, Yuzhi, Lau, Kin Wai, Zhou, Chang
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Artificial neural networks; Computer architecture; convolutional neural networks; Distortion; Feature extraction; Full-reference image quality assessment; Image quality; Linearity; long-range dependencies; Machine learning; Measurement; Model accuracy; Neural networks; Nonlinear distortion; Performance enhancement; Performance evaluation; Quality assessment; quantization; Robustness; spatial pooling; Training
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.2984886

Deep Learning based image quality assessment (IQA) has been shown to greatly improve the quality score prediction accuracy of images with single distortion. However, because these models lack generalizability and the accuracy of multidistortion-based image data is relatively low, designing reliable IQA systems is still an open issue. In this paper, we propose to introduce long-range dependencies between local artifacts and high-order spatial pooling into a convolutional neural network (CNN) model to improve the performance and generalizability of the full-reference IQA (FR-IQA). This long-range dependencies model is based on the hypothesis that local apparent artifacts can affect the overall image quality. The proposed network architecture adopts a non-local means algorithm to establish connections between all positions in the deep feature space and uses the Minkowski function to improve the non-linearity of the spatial pooling. Based on this architecture, a robust FR-IQA system has been constructed and evaluated on three well-known single-distortion-based IQA databases (LIVE, CSIQ, and TID2013) and a multidistortion-based IQA database (MDID). Experimental results demonstrate that, compared with the latest FR-IQA systems, the proposed long-range dependencies-boosted CNN-based FR-IQA system can achieve state-of-the-art performance. A comprehensive cross-database evaluation also shows that the proposed system is sufficiently generalized between different databases and multidistortion-based image data is more useful for training robust image quality metrics.