Information Propagation by Composited Labels in Natural Language Processing

TL;DR

This paper introduces a novel framework for modeling information flow in NLP by defining labels as mappings between text regions and their broader contexts, enabling analysis of entity linkage and information loss.

Contribution

It proposes a new conceptual approach to entity labeling in NLP that captures information propagation and quantifies information loss using entropy-based measures.

Findings

Entities form a graph based on inclusion relations.

Information loss can be measured by entropy.

The framework models information flow in text regions.

Abstract

In natural language processing (NLP), labeling on regions of text, such as words, sentences and paragraphs, is a basic task. In this paper, label is defined as map between mention of entity in a region on text and context of entity in a broader region on text containing the mention. This definition naturally introduces linkage of entities induced from inclusion relation of regions, and connected entities form a graph representing information flow defined by map. It also enables calculation of information loss through map using entropy, and entropy lost is regarded as distance between two entities over a path on graph.