A Survey on ML4VIS: Applying Machine Learning Advances to Data Visualization

• Published in: IEEE transactions on visualization and computer graphics Vol. 28; no. 12; pp. 5134 - 5153
• Main Authors: Wang, Qianwen, Zhu-Tian, Chen, Wang, Yong, Qu, Huamin
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.12.2022; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Analytical models; Cognitive tasks; Data mining; Data visualization; Guidelines; Human computer interaction; Machine learning; Mapping; ML4VIS; Scientific visualization; survey; Visualization
• ISSN: 1077-2626; 1941-0506; 1941-0506
• DOI: 10.1109/TVCG.2021.3106142

Inspired by the great success of machine learning (ML), researchers have applied ML techniques to visualizations to achieve a better design, development, and evaluation of visualizations. This branch of studies, known as ML4VIS, is gaining increasing research attention in recent years. To successfully adapt ML techniques for visualizations, a structured understanding of the integration of ML4VIS is needed. In this article, we systematically survey 88 ML4VIS studies, aiming to answer two motivating questions: "what visualization processes can be assisted by ML?" and "how ML techniques can be used to solve visualization problems? " This survey reveals seven main processes where the employment of ML techniques can benefit visualizations: Data Processing4VIS, Data-VIS Mapping, Insight Communication, Style Imitation, VIS Interaction, VIS Reading, and User Profiling . The seven processes are related to existing visualization theoretical models in an ML4VIS pipeline, aiming to illuminate the role of ML-assisted visualization in general visualizations. Meanwhile, the seven processes are mapped into main learning tasks in ML to align the capabilities of ML with the needs in visualization. Current practices and future opportunities of ML4VIS are discussed in the context of the ML4VIS pipeline and the ML-VIS mapping. While more studies are still needed in the area of ML4VIS, we hope this article can provide a stepping-stone for future exploration. A web-based interactive browser of this survey is available at https://ml4vis.github.io .