Impact of Task Phrasing on Presumptions in Large Language Models

TL;DR

This study explores how task phrasing influences presumptions in large language models, affecting their decision-making and reasoning, with implications for safety and reliability in real-world applications.

Contribution

It demonstrates that neutral task phrasing reduces presumptions in LLMs, improving their logical reasoning and adaptability in decision-making tasks.

Findings

LLMs are susceptible to presumptions even with reasoning steps

Neutral phrasing leads to less presumptive and more logical reasoning

Proper task phrasing is crucial for safe and reliable LLM deployment

Abstract

Concerns with the safety and reliability of applying large-language models (LLMs) in unpredictable real-world applications motivate this study, which examines how task phrasing can lead to presumptions in LLMs, making it difficult for them to adapt when the task deviates from these assumptions. We investigated the impact of these presumptions on the performance of LLMs using the iterated prisoner's dilemma as a case study. Our experiments reveal that LLMs are susceptible to presumptions when making decisions even with reasoning steps. However, when the task phrasing was neutral, the models demonstrated logical reasoning without much presumptions. These findings highlight the importance of proper task phrasing to reduce the risk of presumptions in LLMs.