TrustDataFilter:Leveraging Trusted Knowledge Base Data for More Effective Filtering of Unknown Information

TL;DR

TrustDataFilter leverages trusted knowledge bases and natural language inference models to enhance the accuracy and consistency of filtering unknown information in domain-specific knowledge base construction.

Contribution

The paper introduces the self-nli-TDF framework that uses large language models for trustworthiness assessment and reasoning, improving filtering performance in knowledge base construction.

Findings

Improved filtering quality with more consistent results

Effective use of large language models like RoBERTa and GPT-3.5

Validated on datasets from biology, radiation, and science domains

Abstract

With the advancement of technology and changes in the market, the demand for the construction of domain-specific knowledge bases has been increasing, either to improve model performance or to promote enterprise innovation and competitiveness. The construction of domain-specific knowledge bases typically relies on web crawlers or existing industry databases, leading to problems with accuracy and consistency of the data. To address these challenges, we considered the characteristics of domain data, where internal knowledge is interconnected, and proposed the Self-Natural Language Inference Data Filtering (self-nli-TDF) framework. This framework compares trusted filtered knowledge with the data to be filtered, deducing the reasoning relationship between them, thus improving filtering performance. The framework uses plug-and-play large language models for trustworthiness assessment and employs the RoBERTa-MNLI model from the NLI domain for reasoning. We constructed three datasets in the domains of biology, radiation, and science, and conducted experiments using RoBERTa, GPT3.5, and the local Qwen2 model. The experimental results show that this framework improves filter quality, producing more consistent and reliable filtering results.