Occurrence Statistics of Entities, Relations and Types on the Web

TL;DR

This paper discusses the challenge of estimating entity occurrences on the web due to distribution mismatches and proposes using maximum mean discrepancy for better estimation, reviewing related disambiguation techniques.

Contribution

It introduces the application of maximum mean discrepancy to improve occurrence statistics estimation of entities on the web, addressing distribution mismatch issues.

Findings

Maximum mean discrepancy effectively estimates entity occurrence statistics.

Web entity distributions differ significantly from training data.

Review of named entity disambiguation techniques.

Abstract

The problem of collecting reliable estimates of occurrence of entities on the open web forms the premise for this report. The models learned for tagging entities cannot be expected to perform well when deployed on the web. This is owing to the severe mismatch in the distributions of such entities on the web and in the relatively diminutive training data. In this report, we build up the case for maximum mean discrepancy for estimation of occurrence statistics of entities on the web, taking a review of named entity disambiguation techniques and related concepts along the way.