Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models

TL;DR

This paper investigates how large language models struggle with redefinition tasks involving physical constants, revealing performance degradation and increased false confidence as models scale, despite prompting strategies.

Contribution

It introduces the redefinition task to evaluate reasoning gaps in LLMs and demonstrates that scaling worsens performance and confidence issues despite various prompting methods.

Findings

Model performance degrades with scale.

False confidence increases as models grow larger.

Prompting strategies influence responses but do not eliminate anchoring to memorized values.

Abstract

Inverse tasks can uncover potential reasoning gaps as Large Language Models (LLMs) scale up. In this work, we explore the redefinition task, in which we assign alternative values to well-known physical constants and units of measure, prompting LLMs to respond accordingly. Our findings show that not only does model performance degrade with scale, but its false confidence also rises. Moreover, while factors such as prompting strategies or response formatting are influential, they do not preclude LLMs from anchoring to memorized values.