A study on quality metrics vs. human perception: Can visual measures help us to filter visualizations of interest?

• Published in: Information technology (Munich, Germany) Vol. 57; no. 1; pp. 11 - 21
• Main Authors: Lehmann, Dirk J., Hundt, Sebastian, Theisel, Holger
• Format: Journal Article
• Language: English
• Published: De Gruyter Oldenbourg 28.02.2015
• Subjects: Human-centered computing→Visualization→Empirical studies in visualization; perception study; Quality metric; visualization
• ISSN: 1611-2776; 2196-7032
• DOI: 10.1515/itit-2014-1070

The number of visualizations being required for a complete view on data non-linearly grows with the number of data dimensions. Thus, relevant visualizations need to be filtered to guide the user during the visual search. A popular filter approach is the usage of quality metrics, which map a visual pattern to a real number. This way, visualizations that contain interesting patterns are automatically detected. Quality metrics are a useful tool in visual analysis, if they resemble the human perception. In this work we present a broad study to examine the relation between filtering relevant visualizations based on human perception versus quality metrics. For this, seven widely-used quality metrics were tested on five high-dimensional datasets, covering scatterplots, parallel coordinates, and radial visualizations. In total, 102 participants were available. The results of our studies show that quality metrics often work similar to the human perception. Interestingly, a subset of so-called Scagnostic measures does the best job.