SG-FSM: A Self-Guiding Zero-Shot Prompting Paradigm for Multi-Hop Question Answering Based on Finite State Machine

TL;DR

SG-FSM introduces a novel self-guiding finite state machine approach that enhances multi-hop question answering in large language models by iteratively breaking down questions and correcting errors, leading to improved accuracy and reduced hallucinations.

Contribution

This paper presents SG-FSM, a new prompting paradigm that models multi-hop reasoning as a finite state machine, improving accuracy and robustness over traditional chain-of-thought methods.

Findings

Outperforms strong baselines on multiple MHQA benchmarks.

Reduces hallucination and error propagation in multi-step reasoning.

Enhances adherence to output formats, simplifying evaluation.

Abstract

Large Language Models with chain-of-thought prompting, such as OpenAI-o1, have shown impressive capabilities in natural language inference tasks. However, Multi-hop Question Answering (MHQA) remains challenging for many existing models due to issues like hallucination, error propagation, and limited context length. To address these challenges and enhance LLMs' performance on MHQA, we propose the Self-Guiding prompting Finite State Machine (SG-FSM), designed to strengthen multi-hop reasoning abilities. Unlike traditional chain-of-thought methods, SG-FSM tackles MHQA by iteratively breaking down complex questions into sub-questions, correcting itself to improve accuracy. It processes one sub-question at a time, dynamically deciding the next step based on the current context and results, functioning much like an automaton. Experiments across various benchmarks demonstrate the effectiveness of our approach, outperforming strong baselines on challenging datasets such as Musique. SG-FSM reduces hallucination, enabling recovery of the correct final answer despite intermediate errors. It also improves adherence to specified output formats, simplifying evaluation significantly.