SAKR: Enhancing Retrieval-Augmented Generation via Streaming Algorithm and K-Means Clustering

TL;DR

This paper introduces SAKR, a novel retrieval-augmented generation method that combines streaming algorithms and k-means clustering to improve memory efficiency and update speed in large-scale, streaming data environments.

Contribution

It presents a new approach integrating streaming algorithms and k-means clustering into RAG to dynamically update indexes and reduce memory usage while maintaining accuracy.

Findings

Outperforms traditional RAG in accuracy on large-scale data

Reduces memory consumption significantly

Speeds up query processing with clustering

Abstract

Retrieval-augmented generation (RAG) has achieved significant success in information retrieval to assist large language models LLMs because it builds an external knowledge database. However, it also has many problems, it consumes a lot of memory because of the enormous database, and it cannot update the established index database in time when confronted with massive streaming data. To reduce the memory required for building the database and maintain accuracy simultaneously, we proposed a new approach integrating a streaming algorithm with k-means clustering into RAG. Our approach applied a streaming algorithm to update the index dynamically and reduce memory consumption. Additionally, the k-means algorithm clusters highly similar documents, and the query time would be shortened. We conducted comparative experiments on four methods, and the results indicated that RAG with streaming algorithm and k-means clusters outperforms traditional RAG in accuracy and memory, particularly when dealing with large-scale streaming data.