The What-If Tool: Interactive Probing of Machine Learning Models

• Published in: IEEE transactions on visualization and computer graphics Vol. 26; no. 1; pp. 56 - 65
• Main Authors: Wexler, James, Pushkarna, Mahima, Bolukbasi, Tolga, Wattenberg, Martin, Viegas, Fernanda, Wilson, Jimbo
• Format: Journal Article
• Language: English
• Published: United States IEEE 01.01.2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Analytical models; Artificial intelligence; Computational modeling; Computer Graphics; Computer Simulation; Data models; Data visualization; Databases, Factual; Humans; Interactive Machine Learning; Machine Learning; Model Comparison; Model Debugging; Predictive models; Software; Source code; User-Computer Interface
• ISSN: 1077-2626; 1941-0506; 1941-0506
• DOI: 10.1109/TVCG.2019.2934619

A key challenge in developing and deploying Machine Learning (ML) systems is understanding their performance across a wide range of inputs. To address this challenge, we created the What-If Tool, an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding. The What-If Tool lets practitioners test performance in hypothetical situations, analyze the importance of different data features, and visualize model behavior across multiple models and subsets of input data. It also lets practitioners measure systems according to multiple ML fairness metrics. We describe the design of the tool, and report on real-life usage at different organizations.