Subjective and Objective Quality Assessment of Rendered Human Avatar Videos in Virtual Reality

• Published in: IEEE transactions on image processing Vol. 33; pp. 5740 - 5754
• Main Authors: Chen, Yu-Chih, Saha, Avinab, Chapiro, Alexandre, Hane, Christian, Bazin, Jean-Charles, Qiu, Bo, Zanetti, Stefano, Katsavounidis, Ioannis, Bovik, Alan C.
• Format: Journal Article
• Language: English
• Published: United States IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: 3D mesh; Algorithms; Augmented reality; Avatar; Avatars; human avatar video; Human performance; Humans; Image coding; Image Processing, Computer-Assisted - methods; Monitoring; Prediction models; Predictive models; Quality assessment; six degrees of freedom; Solid modeling; State-of-the-art reviews; Three-dimensional displays; Video compression; video quality assessment; Video Recording - methods; Video transmission; Videos; Virtual humans; Virtual Reality; Visualization
• ISSN: 1057-7149; 1941-0042; 1941-0042
• DOI: 10.1109/TIP.2024.3468881

We study the visual quality judgments of human subjects on digital human avatars (sometimes referred to as "holograms" in the parlance of virtual reality [VR] and augmented reality [AR] systems) that have been subjected to distortions. We also study the ability of video quality models to predict human judgments. As streaming human avatar videos in VR or AR become increasingly common, the need for more advanced human avatar video compression protocols will be required to address the tradeoffs between faithfully transmitting high-quality visual representations while adjusting to changeable bandwidth scenarios. During transmission over the internet, the perceived quality of compressed human avatar videos can be severely impaired by visual artifacts. To optimize trade-offs between perceptual quality and data volume in practical workflows, video quality assessment (VQA) models are essential tools. However, there are very few VQA algorithms developed specifically to analyze human body avatar videos, due, at least in part, to the dearth of appropriate and comprehensive datasets of adequate size. Towards filling this gap, we introduce the LIVE-Meta Rendered Human Avatar VQA Database, which contains 720 human avatar videos processed using 20 different combinations of encoding parameters, labeled by corresponding human perceptual quality judgments that were collected in six degrees of freedom VR headsets. To demonstrate the usefulness of this new and unique video resource, we use it to study and compare the performances of a variety of state-of-the-art Full Reference and No Reference video quality prediction models, including a new model called HoloQA. As a service to the research community, we publicly releases the metadata of the new database at https://live.ece.utexas.edu/research/LIVE-Meta-rendered-human-avatar/index.html .