Evaluating AI Evaluation: Perils and Prospects

TL;DR

This paper critiques current AI evaluation methods, highlighting their inadequacies, and advocates for cognitively-inspired approaches to improve assessment of AI systems' safety and capabilities.

Contribution

It proposes integrating cognitive science principles into AI evaluation and analyzes emerging 'Evals' to enhance the rigor and safety of AI assessments.

Findings

Current evaluation methods are fundamentally inadequate.

Cognitively-inspired approaches offer promising pathways.

Emerging 'Evals' can be refined for better AI assessment.

Abstract

As AI systems appear to exhibit ever-increasing capability and generality, assessing their true potential and safety becomes paramount. This paper contends that the prevalent evaluation methods for these systems are fundamentally inadequate, heightening the risks and potential hazards associated with AI. I argue that a reformation is required in the way we evaluate AI systems and that we should look towards cognitive sciences for inspiration in our approaches, which have a longstanding tradition of assessing general intelligence across diverse species. We will identify some of the difficulties that need to be overcome when applying cognitively-inspired approaches to general-purpose AI systems and also analyse the emerging area of "Evals". The paper concludes by identifying promising research pathways that could refine AI evaluation, advancing it towards a rigorous scientific domain that contributes to the development of safe AI systems.