Early Discovery of Disappearing Entities in Microblogs

TL;DR

This paper presents a method for early detection of disappearing entities in microblogs, using time-sensitive distant supervision and refined embeddings, achieving over one month lead time compared to Wikipedia updates.

Contribution

It introduces a novel approach combining time-sensitive distant supervision and embedding refinement for timely detection of disappearing entities in microblogs.

Findings

Over 70% of disappearing entities are detected earlier than Wikipedia updates.

The average lead time for detection exceeds one month.

The method is effective on datasets in both English and Japanese.

Abstract

We make decisions by reacting to changes in the real world, in particular, the emergence and disappearance of impermanent entities such as events, restaurants, and services. Because we want to avoid missing out on opportunities or making fruitless actions after they have disappeared, it is important to know when entities disappear as early as possible. We thus tackle the task of detecting disappearing entities from microblogs, whose posts mention various entities, in a timely manner. The major challenge is detecting uncertain contexts of disappearing entities from noisy microblog posts. To collect these disappearing contexts, we design time-sensitive distant supervision, which utilizes entities from the knowledge base and time-series posts, for this task to build large-scale Twitter datasets\footnote{We will release the datasets (tweet IDs) used in the experiments to promote reproducibility.} for English and Japanese. To ensure robust detection in noisy environments, we refine pretrained word embeddings of the detection model on microblog streams of the target day. Experimental results on the Twitter datasets confirmed the effectiveness of the collected labeled data and refined word embeddings; more than 70\% of the detected disappearing entities in Wikipedia are discovered earlier than the update on Wikipedia, and the average lead-time is over one month.