NarrativeTime: Dense Temporal Annotation on a Timeline

TL;DR

NarrativeTime introduces a timeline-based framework for dense temporal annotation, enabling full coverage of event pairs and significantly increasing annotation density compared to prior sparse methods.

Contribution

It is the first framework to achieve complete temporal coverage on a timeline, with new annotated corpus, tools, and analysis for dense temporal annotation.

Findings

Full re-annotation of TimeBankDense shows comparable agreement with increased density.

TimeBankNT corpus provides fully annotated texts by expert annotators.

Open-source tools facilitate annotation and conversion to TimeML format.

Abstract

For the past decade, temporal annotation has been sparse: only a small portion of event pairs in a text was annotated. We present NarrativeTime, the first timeline-based annotation framework that achieves full coverage of all possible TLinks. To compare with the previous SOTA in dense temporal annotation, we perform full re-annotation of TimeBankDense corpus, which shows comparable agreement with a significant increase in density. We contribute TimeBankNT corpus (with each text fully annotated by two expert annotators), extensive annotation guidelines, open-source tools for annotation and conversion to TimeML format, baseline results, as well as quantitative and qualitative analysis of inter-annotator agreement.