Improving governance outcomes through AI documentation: Bridging theory and practice

• Published in: arXiv.org
• Main Authors: Winecoff, Amy A, Bogen, Miranda
• Format: Paper
• Language: English
• Published: Ithaca Cornell University Library, arXiv.org 13.09.2024
• Subjects: Best practice; Documentation; Empirical analysis; Organizations
• ISSN: 2331-8422

Documentation plays a crucial role in both external accountability and internal governance of AI systems. Although there are many proposals for documenting AI data, models, systems, and methods, the ways these practices enhance governance as well as the challenges practitioners and organizations face with documentation remain underexplored. In this paper, we analyze 37 proposed documentation frameworks and 21 empirical studies evaluating their use. We identify potential hypotheses about how documentation can strengthen governance, such as informing stakeholders about AI risks and usage, fostering collaboration, encouraging ethical reflection, and reinforcing best practices. However, empirical evidence shows that practitioners often encounter obstacles that prevent documentation from achieving these goals. We also highlight key considerations for organizations when designing documentation, such as determining the appropriate level of detail and balancing automation in the process. Finally, we offer recommendations for further research and for implementing effective documentation practices in real-world contexts.