Reasoning with RAGged events: RAG-Enhanced Event Knowledge Base Construction and reasoning with proof-assistants
Stergios Chatzikyriakidis

TL;DR
This paper develops models for extracting structured historical event data from texts using LLMs with enhancement strategies, and translates these into formal Coq representations for advanced reasoning and validation.
Contribution
It introduces a novel pipeline combining LLM-based event extraction with RAG enhancements and automatic translation into Coq for higher-order reasoning.
Findings
Base generation yields broad coverage with Claude and GPT-4.
RAG improves precision and metadata completeness.
Model architecture influences enhancement effectiveness.
Abstract
Extracting structured computational representations of historical events from narrative text remains computationally expensive when constructed manually. While RDF/OWL reasoners enable graph-based reasoning, they are limited to fragments of first-order logic, preventing deeper temporal and semantic analysis. This paper addresses both challenges by developing automatic historical event extraction models using multiple LLMs (GPT-4, Claude, Llama 3.2) with three enhancement strategies: pure base generation, knowledge graph enhancement, and Retrieval-Augmented Generation (RAG). We conducted comprehensive evaluations using historical texts from Thucydides. Our findings reveal that enhancement strategies optimize different performance dimensions rather than providing universal improvements. For coverage and historical breadth, base generation achieves optimal performance with Claude and GPT-4…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Semantic Web and Ontologies · Biomedical Text Mining and Ontologies
