Why No Reference Metrics for Image and Video Quality Lack Accuracy and Reproducibility
This article provides a comprehensive overview of no reference (NR) metrics for image quality analysis (IQA) and video quality analysis (VQA). We examine 26 independent evaluations of NR metrics (previously published) and analyze 32 NR metrics on six IQA datasets and six VQA datasets (new results)....
Saved in:
Published in | IEEE transactions on broadcasting Vol. 69; no. 1; pp. 97 - 117 |
---|---|
Main Author | |
Format | Journal Article |
Language | English |
Published |
New York
IEEE
01.03.2023
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Summary: | This article provides a comprehensive overview of no reference (NR) metrics for image quality analysis (IQA) and video quality analysis (VQA). We examine 26 independent evaluations of NR metrics (previously published) and analyze 32 NR metrics on six IQA datasets and six VQA datasets (new results). Where NR metric developers claim Pearson correlation values between 0.66 and 0.99, our measurements range from 0.0 to 0.63. None of the NR metrics we analyzed are accurate enough to be deployed by industry. Performance evaluations that indicate otherwise are based on insufficient data and highly inaccurate. We will examine development strategies, tools, datasets, root cause analysis, and our baseline metric for collaboration, Sawatch. |
---|---|
ISSN: | 0018-9316 1557-9611 |
DOI: | 10.1109/TBC.2022.3191059 |