To Err is AI : A Case Study Informing LLM Flaw Reporting Practices

TL;DR

This paper analyzes a large-scale bug bounty involving 495 hackers to identify best practices in safety flaw reporting for large language models, aiming to improve model safety and transparency.

Contribution

It introduces a set of lessons learned and best practices for flaw reporting processes, safety documentation, and staffing to enhance LLM safety and incident prevention.

Findings

Effective flaw reporting practices improve safety transparency.

Structured safety documentation aids in incident reduction.

Dedicated safety staffing enhances flaw identification.

Abstract

In August of 2024, 495 hackers generated evaluations in an open-ended bug bounty targeting the Open Language Model (OLMo) from The Allen Institute for AI. A vendor panel staffed by representatives of OLMo's safety program adjudicated changes to OLMo's documentation and awarded cash bounties to participants who successfully demonstrated a need for public disclosure clarifying the intent, capacities, and hazards of model deployment. This paper presents a collection of lessons learned, illustrative of flaw reporting best practices intended to reduce the likelihood of incidents and produce safer large language models (LLMs). These include best practices for safety reporting processes, their artifacts, and safety program staffing.