HYBRIDMIND: Meta Selection of Natural Language and Symbolic Language for Enhanced LLM Reasoning

TL;DR

HYBRIDMIND introduces an adaptive method that dynamically chooses between natural language and symbolic reasoning for large language models, significantly improving reasoning accuracy across various tasks.

Contribution

The paper presents HYBRIDMIND, a novel meta-selection strategy that enhances LLM reasoning by adaptively choosing the optimal reasoning approach, outperforming existing methods.

Findings

Fine-tuning LLaMA-3.1-8B-Instruct improves reasoning accuracy by 4.4% on FOLIO.

GPT-3.5-turbo as a meta-selector yields 10% better results on FOLIO's challenging subset.

HYBRIDMIND outperforms pure natural language reasoning approaches.

Abstract

LLMs approach logical and mathematical reasoning through natural or symbolic languages. While natural language offers human-accessible flexibility but suffers from ambiguity, symbolic reasoning provides precise, machine-executable inferences at the cost of strict domain constraints. We introduce HYBRIDMIND, an adaptive strategy that selects the optimal reasoning approach for each reasoning problem. Through extensive experiments, we evaluate both prompting-based approaches with state-of-the-art LLMs and fine-tuned open-source models. We find that fine-tuning LLaMA-3.1-8B-Instruct as a meta-selector outperforms GPT-4o's natural language reasoning by 4.4\% on FOLIO and 1.3\% on MATH. More notably, using GPT-3.5-turbo as a prompted meta-selector yields a 10\% improvement on FOLIO's challenging subset compared to GPT-4o. We will release our code and data to support future research.