NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals
Jaden Fiotto-Kaufman, Alexander R. Loftus, Eric Todd, Jannik, Brinkmann, Koyena Pal, Dmitrii Troitskii, Michael Ripa, Adam Belfki, Can, Rager, Caden Juang, Aaron Mueller, Samuel Marks, Arnab Sen Sharma, Francesca, Lucchetti, Nikhil Prakash, Carla Brodley, Arjun Guha

TL;DR
This paper presents NNsight and NDIF, open-source tools that facilitate scientific analysis of large neural network internals by providing scalable, transparent access without the need for hosting models locally.
Contribution
The authors introduce NNsight and NDIF, novel open-source systems that enable efficient, scalable, and transparent study of large neural networks' internals, bridging a significant gap in AI research.
Findings
Enabled new research methods on large models
Demonstrated scalable inference with NDIF
Benchmarked performance against previous approaches
Abstract
We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks. NNsight is an open-source system that extends PyTorch to introduce deferred remote execution. The National Deep Inference Fabric (NDIF) is a scalable inference service that executes NNsight requests, allowing users to share GPU resources and pretrained models. These technologies are enabled by the Intervention Graph, an architecture developed to decouple experimental design from model runtime. Together, this framework provides transparent and efficient access to the internals of deep neural networks such as very large language models (LLMs) without imposing the cost or complexity of hosting customized models individually. We conduct a quantitative survey of the machine learning literature that reveals a growing gap in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsICT Impact and Policies
Methodstravel james
