Towards Publicly Accountable Frontier LLMs: Building an External Scrutiny Ecosystem under the ASPIRE Framework
Markus Anderljung, Everett Thornton Smith, Joe O'Brien, Lisa Soder,, Benjamin Bucknall, Emma Bluemke, Jonas Schuett, Robert Trager, Lacey Strahm,, Rumman Chowdhury

TL;DR
This paper advocates for a comprehensive external scrutiny ecosystem for frontier LLMs, emphasizing the importance of transparency, independence, and resources to ensure trustworthy deployment and societal accountability.
Contribution
It introduces the ASPIRE framework outlining six key requirements for effective external scrutiny of frontier LLMs and discusses its application across the AI lifecycle.
Findings
Six requirements for external scrutiny are identified and organized under the ASPIRE framework.
External scrutiny can be integrated throughout the AI lifecycle to enhance accountability.
Recommendations are provided for policymakers to support external evaluation efforts.
Abstract
With the increasing integration of frontier large language models (LLMs) into society and the economy, decisions related to their training, deployment, and use have far-reaching implications. These decisions should not be left solely in the hands of frontier LLM developers. LLM users, civil society and policymakers need trustworthy sources of information to steer such decisions for the better. Involving outside actors in the evaluation of these systems - what we term 'external scrutiny' - via red-teaming, auditing, and external researcher access, offers a solution. Though there are encouraging signs of increasing external scrutiny of frontier LLMs, its success is not assured. In this paper, we survey six requirements for effective external scrutiny of frontier AI systems and organize them under the ASPIRE framework: Access, Searching attitude, Proportionality to the risks, Independence,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Artificial Intelligence in Healthcare and Education
