Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators

TL;DR

This paper assesses the openness, transparency, and accountability of instruction-tuned large language models, highlighting disparities in openness levels and emphasizing the importance of scientific documentation for responsible AI development.

Contribution

It provides a systematic evaluation of open-source LLM projects, documenting degrees of openness and revealing gaps in transparency and scientific rigor.

Findings

Many open-source projects use undocumented or legally dubious data

Few projects share instruction-tuning data or detailed documentation

Openness levels impact fairness and accountability in AI models

Abstract

Large language models that exhibit instruction-following behaviour represent one of the biggest recent upheavals in conversational interfaces, a trend in large part fuelled by the release of OpenAI's ChatGPT, a proprietary large language model for text generation fine-tuned through reinforcement learning from human feedback (LLM+RLHF). We review the risks of relying on proprietary software and survey the first crop of open-source projects of comparable architecture and functionality. The main contribution of this paper is to show that openness is differentiated, and to offer scientific documentation of degrees of openness in this fast-moving field. We evaluate projects in terms of openness of code, training data, model weights, RLHF data, licensing, scientific documentation, and access methods. We find that while there is a fast-growing list of projects billing themselves as 'open source', many inherit undocumented data of dubious legality, few share the all-important instruction-tuning (a key site where human annotation labour is involved), and careful scientific documentation is exceedingly rare. Degrees of openness are relevant to fairness and accountability at all points, from data collection and curation to model architecture, and from training and fine-tuning to release and deployment.