NNsight and NDIF: Democratizing Access to Foundation Model Internals

The enormous scale of state-of-the-art foundation models has limited their accessibility to scientists, because customized experiments at large model sizes require costly hardware and complex engineering that is impractical for most researchers. To alleviate these problems, we introduce NNsight, an...

Full description

Saved in:
Bibliographic Details
Published inarXiv.org
Main Authors Fiotto-Kaufman, Jaden, Loftus, Alexander R, Todd, Eric, Brinkmann, Jannik, Juang, Caden, Pal, Koyena, Rager, Can, Mueller, Aaron, Marks, Samuel, Arnab Sen Sharma, Lucchetti, Francesca, Ripa, Michael, Belfki, Adam, Prakash, Nikhil, Multani, Sumeet, Brodley, Carla, Guha, Arjun, Bell, Jonathan, Wallace, Byron, Bau, David
Format Paper
LanguageEnglish
Published Ithaca Cornell University Library, arXiv.org 18.07.2024
Subjects
Application programming interface
Python
Source code
Online AccessGet full text

Cover

More Information
Summary:The enormous scale of state-of-the-art foundation models has limited their accessibility to scientists, because customized experiments at large model sizes require costly hardware and complex engineering that is impractical for most researchers. To alleviate these problems, we introduce NNsight, an open-source Python package with a simple, flexible API that can express interventions on any PyTorch model by building computation graphs. We also introduce NDIF, a collaborative research platform providing researchers access to foundation-scale LLMs via the NNsight API. Code, documentation, and tutorials are available at https://www.nnsight.net.
ISSN:2331-8422