Tree-Based Text Retrieval via Hierarchical Clustering in RAGFrameworks: Application on Taiwanese Regulations

TL;DR

This paper introduces a hierarchical clustering-based retrieval method for RAG systems that adaptively selects relevant documents without predefining k, improving accuracy and interpretability in legal text retrieval.

Contribution

The proposed method eliminates the need to predefine k in RAG retrieval, enhancing relevance and interpretability in legal document retrieval tasks.

Findings

Achieves superior performance in expert evaluations

Maintains high precision without predefining k

Improves accuracy and interpretability in legal text retrieval

Abstract

Traditional Retrieval-Augmented Generation (RAG) systems employ brute-force inner product search to retrieve the top-k most similar documents, then combined with the user query and passed to a language model. This allows the model to access external knowledge and reduce hallucinations. However, selecting an appropriate k value remains a significant challenge in practical applications: a small k may fail to retrieve sufficient information, while a large k can introduce excessive and irrelevant content. To address this, we propose a hierarchical clustering-based retrieval method that eliminates the need to predefine k. Our approach maintains the accuracy and relevance of system responses while adaptively selecting semantically relevant content. In the experiment stage, we applied our method to a Taiwanese legal dataset with expert-graded queries. The results show that our approach achieves superior performance in expert evaluations and maintains high precision while eliminating the need to predefine k, demonstrating improved accuracy and interpretability in legal text retrieval tasks. Our framework is simple to implement and easily integrates with existing RAG pipelines, making it a practical solution for real-world applications under limited resources.