LLMs as Method Actors: A Model for Prompt Engineering and Architecture

TL;DR

This paper introduces the "Method Actors" mental model for prompt engineering, significantly improving LLM performance on complex reasoning tasks like the Connections puzzle by framing models as actors following scripts.

Contribution

The paper proposes the "Method Actors" approach as a novel prompt architecture that enhances LLM reasoning and problem-solving capabilities beyond existing methods.

Findings

Method Actors approach solves 86% of puzzles, outperforming Chain of Thoughts.

OpenAI's o1-preview model solves 100% of puzzles with iterative prompting.

Method Actor prompts increase perfect solution rate from 76% to 87% for o1-preview.

Abstract

We introduce "Method Actors" as a mental model for guiding LLM prompt engineering and prompt architecture. Under this mental model, LLMs should be thought of as actors; prompts as scripts and cues; and LLM responses as performances. We apply this mental model to the task of improving LLM performance at playing Connections, a New York Times word puzzle game that prior research identified as a challenging benchmark for evaluating LLM reasoning. Our experiments with GPT-4o show that a "Method Actors" approach can significantly improve LLM performance over both a vanilla and "Chain of Thoughts" approach. A vanilla approach solves 27% of Connections puzzles in our dataset and a "Chain of Thoughts" approach solves 41% of puzzles, whereas our strongest "Method Actor" approach solves 86% of puzzles. We also test OpenAI's newest model designed specifically for complex reasoning tasks, o1-preview. When asked to solve a puzzle all at once, o1-preview solves 79% of Connections puzzles in our dataset, and when allowed to build puzzle solutions one guess at a time over multiple API calls, o1-preview solves 100% of the puzzles. Incorporating a "Method Actor" prompt architecture increases the percentage of puzzles that o1-preview solves perfectly from 76% to 87%.