IEPile: Unearthing Large-Scale Schema-Based Information Extraction Corpus

TL;DR

IEPile is a large-scale bilingual instruction corpus designed to improve information extraction capabilities of LLMs, addressing the limitations of existing datasets by providing standardized schema-based data.

Contribution

The paper introduces IEPile, a comprehensive bilingual IE dataset with schema-based instructions, significantly expanding data scale and quality for LLM training.

Findings

Enhanced LLM performance in IE tasks

Improved zero-shot generalization capabilities

Open-sourced dataset and models for community use

Abstract

Large Language Models (LLMs) demonstrate remarkable potential across various domains; however, they exhibit a significant performance gap in Information Extraction (IE). Note that high-quality instruction data is the vital key for enhancing the specific capabilities of LLMs, while current IE datasets tend to be small in scale, fragmented, and lack standardized schema. To this end, we introduce IEPile, a comprehensive bilingual (English and Chinese) IE instruction corpus, which contains approximately 0.32B tokens. We construct IEPile by collecting and cleaning 33 existing IE datasets, and introduce schema-based instruction generation to unearth a large-scale corpus. Experimentally, IEPile enhance the performance of LLMs for IE, with notable improvements in zero-shot generalization. We open-source the resource and pre-trained models, hoping to provide valuable support to the NLP community.