PaperHelper: Knowledge-Based LLM QA Paper Reading Assistant

TL;DR

PaperHelper is a knowledge-based LLM QA tool that enhances scientific literature review by reducing hallucinations and improving accuracy using RAG, with a user-friendly interface and strong experimental performance.

Contribution

The paper introduces PaperHelper, a novel LLM-based literature review assistant utilizing RAG and advanced technologies to improve accuracy and usability in scientific paper analysis.

Findings

Achieves an F1 Score of 60.04 with low latency

Outperforms basic RAG model by 7% in F1 score

Effectively reduces hallucinations in LLMs during literature review

Abstract

In the paper, we introduce a paper reading assistant, PaperHelper, a potent tool designed to enhance the capabilities of researchers in efficiently browsing and understanding scientific literature. Utilizing the Retrieval-Augmented Generation (RAG) framework, PaperHelper effectively minimizes hallucinations commonly encountered in large language models (LLMs), optimizing the extraction of accurate, high-quality knowledge. The implementation of advanced technologies such as RAFT and RAG Fusion significantly boosts the performance, accuracy, and reliability of the LLMs-based literature review process. Additionally, PaperHelper features a user-friendly interface that facilitates the batch downloading of documents and uses the Mermaid format to illustrate structural relationships between documents. Experimental results demonstrate that PaperHelper, based on a fine-tuned GPT-4 API, achieves an F1 Score of 60.04, with a latency of only 5.8 seconds, outperforming the basic RAG model by 7\% in F1 Score.