A Formal Framework for Linguistic Annotation

TL;DR

This paper introduces a formal framework called annotation graphs for representing and managing linguistic annotations across various data formats, addressing the lack of standardization in the field.

Contribution

It proposes a unified logical structure for linguistic annotations, enabling interoperability and consistency across different formats and tools.

Findings

Demonstrates a common conceptual core among diverse annotation formats.

Provides a formal framework for constructing and searching annotations.

Ensures compatibility with multiple data structures and formats.

Abstract

`Linguistic annotation' covers any descriptive or analytic notations applied to raw language data. The basic data may be in the form of time functions -- audio, video and/or physiological recordings -- or it may be textual. The added notations may include transcriptions of all sorts (from phonetic features to discourse structures), part-of-speech and sense tagging, syntactic analysis, `named entity' identification, co-reference annotation, and so on. While there are several ongoing efforts to provide formats and tools for such annotations and to publish annotated linguistic databases, the lack of widely accepted standards is becoming a critical problem. Proposed standards, to the extent they exist, have focussed on file formats. This paper focuses instead on the logical structure of linguistic annotations. We survey a wide variety of existing annotation formats and demonstrate a common conceptual core, the annotation graph. This provides a formal framework for constructing, maintaining and searching linguistic annotations, while remaining consistent with many alternative data structures and file formats.