A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms

• Published in: 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) pp. 690 - 704
• Main Authors: Coston, Amanda, Kawakami, Anna, Zhu, Haiyi, Holstein, Ken, Heidari, Hoda
• Format: Conference Proceeding
• Language: English
• Published: IEEE 01.02.2023
• Subjects: algorithmic decision support; algorithmic oversight; Decision making; deliberation; Machine learning; Machine learning algorithms; Prediction algorithms; predictive analytics; Reflection; responsible AI; Social sciences; Task analysis; validity
• DOI: 10.1109/SaTML54575.2023.00050

Recent research increasingly brings to question the appropriateness of using predictive tools in complex, real-world tasks. While a growing body of work has explored ways to improve value alignment in these tools, comparatively less work has centered concerns around the fundamental justifiability of using these tools. This work seeks to center validity considerations in deliberations around whether and how to build data-driven algorithms in high-stakes domains. Toward this end, we translate key concepts from validity theory to predictive algorithms. We apply the lens of validity to re-examine common challenges in problem formulation and data issues that jeopardize the justifiability of using predictive algorithms and connect these challenges to the social science discourse around validity. Our interdisciplinary exposition clarifies how these concepts apply to algorithmic decision making contexts. We demonstrate how these validity considerations could distill into a series of high-level questions intended to promote and document reflections on the legitimacy of the predictive task and the suitability of the data.