Toward Efficient Video Compression Artifact Detection and Removal: A Benchmark Dataset

• Published in: IEEE transactions on multimedia Vol. 26; pp. 10816 - 10827
• Main Authors: Lin, Liqun, Wang, Mingxing, Yang, Jing, Zhang, Keke, Zhao, Tiesong
• Format: Journal Article
• Language: English
• Published: IEEE 2024
• Subjects: Distortion; Encoding; Image coding; Perceivable encoding artifact; Quality assessment; Video compression; video compression artifact removal; video quality assessment
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2024.3414549

Video compression leads to compression artifacts, among which Perceivable Encoding Artifacts (PEAs) degrade user perception. Most of existing state-of-the-art Video Compression Artifact Removal (VCAR) methods indiscriminately process all artifacts, thus leading to over-enhancement in non-PEA regions. Therefore, accurate detection and location of PEAs is crucial. In this paper, we propose the largest-ever Fine-grained PEA database (FPEA). First, we employ the popular video codecs, VVC and AVS3, as well as their common test settings, to generate four types of spatial PEAs (blurring, blocking, ringing and color bleeding) and two types of temporal PEAs (flickering and floating). Second, we design a labeling platform and recruit sufficient subjects to manually locate all the above types of PEAs. Third, we propose a voting mechanism and feature matching to synthesize all subjective labels to obtain the final PEA labels with fine-grained locations. Besides, we also provide Mean Opinion Score (MOS) values of all compressed video sequences. Experimental results show the effectiveness of FPEA database on both VCAR and compressed Video Quality Assessment (VQA). We envision that FPEA database will benefit the future development of VCAR, VQA and perception-aware video encoders. The FPEA database has been made publicly available.