The Self-Execution Benchmark: Measuring LLMs' Attempts to Overcome Their Lack of Self-Execution

TL;DR

This paper introduces the Self-Execution Benchmark to evaluate whether large language models can predict aspects of their own responses, revealing limitations in their self-awareness and reasoning about their behavior.

Contribution

The paper proposes a novel benchmark for assessing LLMs' ability to anticipate their own responses, highlighting a fundamental limitation in their self-representational capabilities.

Findings

Models perform poorly on self-prediction tasks

Increased size does not improve self-prediction performance

Reveals limitations in LLMs' self-awareness and reasoning

Abstract

Large language models (LLMs) are commonly evaluated on tasks that test their knowledge or reasoning abilities. In this paper, we explore a different type of evaluation: whether an LLM can predict aspects of its own responses. Since LLMs lack the ability to execute themselves, we introduce the Self-Execution Benchmark, which measures a model's ability to anticipate properties of its output, such as whether a question will be difficult for it, whether it will refuse to answer, or what kinds of associations it is likely to produce. Our experiments show that models generally perform poorly on this benchmark, and that increased model size or capability does not consistently lead to better performance. These results suggest a fundamental limitation in how LLMs represent and reason about their own behavior.