Thread: A Logic-Based Data Organization Paradigm for How-To Question Answering with Retrieval Augmented Generation

TL;DR

This paper introduces Thread, a new data organization paradigm that structures documents into logical units to improve how-to question answering, significantly enhancing success rates and adaptability over traditional chunk-based methods.

Contribution

The paper proposes Thread, a novel paradigm using logic units for better data organization, enabling more effective handling of how-to questions with improved accuracy and flexibility.

Findings

Thread outperforms existing paradigms by 21-33% in success rate.

Reduces retrieval information by up to 75%.

Demonstrates high adaptability across document formats.

Abstract

Recent advances in retrieval-augmented generation (RAG) have substantially improved question-answering systems, particularly for factoid '5Ws' questions. However, significant challenges remain when addressing '1H' questions, specifically how-to questions, which are integral for decision-making and require dynamic, step-by-step responses. The key limitation lies in the prevalent data organization paradigm, chunk, which commonly divides documents into fixed-size segments, and disrupts the logical coherence and connections within the context. To address this, we propose Thread, a novel data organization paradigm enabling systems to handle how-to questions more effectively. Specifically, we introduce a new knowledge granularity, 'logic unit' (LU), where large language models transform documents into more structured and loosely interconnected LUs. Extensive experiments across both open-domain and industrial settings show that Thread outperforms existing paradigms significantly, improving the success rate of handling how-to questions by 21% to 33%. Additionally, Thread demonstrates high adaptability across diverse document formats, reducing retrieval information by up to 75% compared to chunk, and also shows better generalizability to '5Ws' questions, such as multi-hop questions, outperforming other paradigms.