Reasoning Strategies in Large Language Models: Can They Follow, Prefer, and Optimize?

TL;DR

This paper explores how prompting can influence large language models' reasoning strategies, aiming to improve their logical problem-solving by enabling adaptive strategy selection.

Contribution

It introduces methods to guide LLMs in strategy selection, addressing the limitation of favoring a single reasoning approach and proposing ways to enhance reasoning flexibility.

Findings

No single strategy consistently improves accuracy.

Adaptive strategy selection has potential to enhance performance.

Prompting can influence reasoning strategies.

Abstract

Human reasoning involves different strategies, each suited to specific problems. Prior work shows that large language model (LLMs) tend to favor a single reasoning strategy, potentially limiting their effectiveness in diverse reasoning challenges. In this work, we investigate whether prompting can control LLMs reasoning strategies and assess its impact on logical problem-solving. While our experiments show that no single strategy consistently improves accuracy, performance could be enhanced if models could adaptively choose the optimal strategy. We propose methods to guide LLMs in strategy selection, highlighting new ways to refine their reasoning abilities.