Textured Mesh Quality Assessment: Large-scale Dataset and Deep Learning-based Quality Metric

• Published in: ACM transactions on graphics Vol. 42; no. 3; pp. 1 - 20
• Main Authors: Nehmé, Yana, Delanoy, Johanna, Dupont, Florent, Farrugia, Jean-Philippe, Le Callet, Patrick, Lavoué, Guillaume
• Format: Journal Article
• Language: English
• Published: New York, NY, USA ACM 05.06.2023; Association for Computing Machinery
• Subjects: Computer Science; Computing methodologies; Graphics; Image-based rendering; Mesh models; Neural networks; Perception; Texturing; 3D mesh; subjective quality evaluation; deep learning; objective quality evaluation; crowdsourcing; texture; Computer graphics; visual quality assessment; dataset; perceptual metric; perception; Crowdsourcing; Computer Graphics; 3D Mesh; Perceptual Metric; Visual Quality Assessment; Dataset; Subjective Quality Evaluation; Perception; Texture; Deep Learning; Objective Quality Evaluation
• DOI: 10.1145/3592786
• ISSN: 0730-0301

Over the past decade, three-dimensional (3D) graphics have become highly detailed to mimic the real world, exploding their size and complexity. Certain applications and device constraints necessitate their simplification and/or lossy compression, which can degrade their visual quality. Thus, to ensure the best Quality of Experience, it is important to evaluate the visual quality to accurately drive the compression and find the right compromise between visual quality and data size. In this work, we focus on subjective and objective quality assessment of textured 3D meshes. We first establish a large-scale dataset, which includes 55 source models quantitatively characterized in terms of geometric, color, and semantic complexity, and corrupted by combinations of five types of compression-based distortions applied on the geometry, texture mapping, and texture image of the meshes. This dataset contains over 343k distorted stimuli. We propose an approach to select a challenging subset of 3,000 stimuli for which we collected 148,929 quality judgments from over 4,500 participants in a large-scale crowdsourced subjective experiment. Leveraging our subject-rated dataset, a learning-based quality metric for 3D graphics was proposed. Our metric demonstrates state-of-the-art results on our dataset of textured meshes and on a dataset of distorted meshes with vertex colors. Finally, we present an application of our metric and dataset to explore the influence of distortion interactions and content characteristics on the perceived quality of compressed textured meshes.