Benchmarking Temporal Reasoning and Alignment Across Chinese Dynasties
Zhenglin Wang, Jialong Wu, Pengfei LI, Yong Jiang, Deyu Zhou

TL;DR
This paper introduces Chinese Time Reasoning (CTM), a comprehensive benchmark for evaluating large language models on complex, culturally-grounded temporal reasoning across Chinese dynasties, addressing limitations of previous rule-based benchmarks.
Contribution
The paper presents CTM, a novel benchmark that emphasizes contextual, cross-entity, and culturally-grounded temporal reasoning within Chinese dynastic history, filling gaps in existing evaluation methods.
Findings
LLMs face significant challenges on CTM benchmark.
CTM reveals gaps in current models' temporal reasoning abilities.
Potential for improved models through targeted training on CTM.
Abstract
Temporal reasoning is fundamental to human cognition and is crucial for various real-world applications. While recent advances in Large Language Models have demonstrated promising capabilities in temporal reasoning, existing benchmarks primarily rely on rule-based construction, lack contextual depth, and involve a limited range of temporal entities. To address these limitations, we introduce Chinese Time Reasoning (CTM), a benchmark designed to evaluate LLMs on temporal reasoning within the extensive scope of Chinese dynastic chronology. CTM emphasizes cross-entity relationships, pairwise temporal alignment, and contextualized and culturally-grounded reasoning, providing a comprehensive evaluation. Extensive experimental results reveal the challenges posed by CTM and highlight potential avenues for improvement.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsClassical Studies and Philology · Culture, Economy, and Development Studies
