Modeling Semantic Plausibility by Injecting World Knowledge

TL;DR

This paper introduces a new dataset and approach for recognizing plausible but potentially novel events by injecting world knowledge, highlighting the limitations of distributional models and the benefits of explicit knowledge integration.

Contribution

The paper presents a new crowdsourced dataset for semantic plausibility and demonstrates that injecting manually elicited world knowledge improves model performance.

Findings

Distributional models perform poorly on plausibility tasks.

Injecting world knowledge significantly boosts model accuracy.

The dataset serves as an effective benchmark for future plausibility research.

Abstract

Distributional data tells us that a man can swallow candy, but not that a man can swallow a paintball, since this is never attested. However both are physically plausible events. This paper introduces the task of semantic plausibility: recognizing plausible but possibly novel events. We present a new crowdsourced dataset of semantic plausibility judgments of single events such as "man swallow paintball". Simple models based on distributional representations perform poorly on this task, despite doing well on selection preference, but injecting manually elicited knowledge about entity properties provides a substantial performance boost. Our error analysis shows that our new dataset is a great testbed for semantic plausibility models: more sophisticated knowledge representation and propagation could address many of the remaining errors.