CHEW: A Dataset of CHanging Events in Wikipedia
Hsuvas Borkakoty, Luis Espinosa-Anke

TL;DR
CHEW is a new dataset capturing changing events in Wikipedia text, used to evaluate large language models' ability to understand timelines and detect meaning shifts, revealing their limitations in temporal reasoning.
Contribution
The paper introduces CHEW, a novel dataset for studying temporal changes in Wikipedia, and demonstrates its utility in probing LLMs' timeline understanding and meaning shift detection.
Findings
LLMs struggle with accurate timeline construction despite available temporal info.
CHEW-derived embeddings help identify meaning shifts in text.
CHEW provides a valuable resource for temporal and semantic analysis.
Abstract
We introduce CHEW, a novel dataset of changing events in Wikipedia expressed in naturally occurring text. We use CHEW for probing LLMs for their timeline understanding of Wikipedia entities and events in generative and classification experiments. Our results suggest that LLMs, despite having temporal information available, struggle to construct accurate timelines. We further show the usefulness of CHEW-derived embeddings for identifying meaning shift.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsWikis in Education and Collaboration · Natural Language Processing Techniques · Cancer-related gene regulation
