Model evaluation for extreme risks

TL;DR

This paper emphasizes the importance of specialized model evaluation methods to identify and mitigate extreme risks posed by AI systems, focusing on dangerous capabilities and alignment issues.

Contribution

It introduces the concept of dangerous capability and alignment evaluations as essential tools for responsible AI development and risk mitigation.

Findings

Evaluation methods are crucial for identifying harmful AI capabilities.

Model assessments can inform safer deployment and policy decisions.

Enhanced evaluation frameworks are needed for managing extreme risks.

Abstract

Current approaches to building general-purpose AI systems tend to produce systems with both beneficial and harmful capabilities. Further progress in AI development could lead to capabilities that pose extreme risks, such as offensive cyber capabilities or strong manipulation skills. We explain why model evaluation is critical for addressing extreme risks. Developers must be able to identify dangerous capabilities (through "dangerous capability evaluations") and the propensity of models to apply their capabilities for harm (through "alignment evaluations"). These evaluations will become critical for keeping policymakers and other stakeholders informed, and for making responsible decisions about model training, deployment, and security.