Full-Reference Image Quality Assessment with Transformer and DISTS

• Published in: Mathematics (Basel) Vol. 11; no. 7; p. 1599
• Main Authors: Tsai, Pei-Fen, Peng, Huai-Nan, Liao, Chia-Hung, Yuan, Shyan-Ming
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01.04.2023
• Subjects: Algorithms; Artificial neural networks; Convolutional codes; Data transmission; Datasets; deep image structure and texture similarity (DISTS); Deep learning; ensemble IQA; Food science; full-reference IQA; Generative adversarial networks; Image compression; Image processing; Image quality; image quality assessment (IQA); Image restoration; Image transmission; Internet of Things; Neural networks; Performance evaluation; PIPAL dataset; Quality assessment; transformer IQA; Transformers; Transmission efficiency; Taiwan
• ISSN: 2227-7390; 2227-7390
• DOI: 10.3390/math11071599

To improve data transmission efficiency, image compression is a commonly used method with the disadvantage of accompanying image distortion. There are many image restoration (IR) algorithms, and one of the most advanced algorithms is the generative adversarial network (GAN)-based method with a high correlation to the human visual system (HVS). To evaluate the performance of GAN-based IR algorithms, we proposed an ensemble image quality assessment (IQA) called ATDIQA (Auxiliary Transformer with DISTS IQA) to give weights on multiscale features global self-attention transformers and local features of convolutional neural network (CNN) IQA of DISTS. The result not only performed better on the perceptual image processing algorithms (PIPAL) dataset with images by GAN IR algorithms but also has good model generalization over LIVE and TID2013 as traditional distorted image datasets. The ATDIQA ensemble successfully demonstrates its performance with a high correlation with the human judgment score of distorted images.