One Transform to Compute Them All: Efficient Fusion-Based Full-Reference Video Quality Assessment

• Published in: IEEE transactions on image processing Vol. 33; pp. 509 - 524
• Main Authors: Venkataramanan, Abhinau K., Stejerean, Cosmin, Katsavounidis, Ioannis, Bovik, Alan C.
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Accuracy; Bit rate; Computational modeling; contrast sensitivity; Full-reference video quality assessment; FUNQUE; Image coding; Quality assessment; Solid modeling; Streaming media; Video compression; Visual perception; Visual perception driven algorithms; VMAF
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2023.3345227

The Visual Multimethod Assessment Fusion (VMAF) algorithm has recently emerged as a state-of-the-art approach to video quality prediction, that now pervades the streaming and social media industry. However, since VMAF requires the evaluation of a heterogeneous set of quality models, it is computationally expensive. Given other advances in hardware-accelerated encoding, quality assessment is emerging as a significant bottleneck in video compression pipelines. Towards alleviating this burden, we propose a novel Fusion of Unified Quality Evaluators (FUNQUE) framework, by enabling computation sharing and by using a transform that is sensitive to visual perception to boost accuracy. Further, we expand the FUNQUE framework to define a collection of improved low-complexity fused-feature models that advance the state-of-the-art of video quality performance with respect to both accuracy, by 4.2% to 5.3%, and computational efficiency, by factors of 3.8 to 11 times!