Plausible-Parrots @ MSP2023: Enhancing Semantic Plausibility Modeling using Entity and Event Knowledge

TL;DR

This paper enhances large language models with external entity and event knowledge to improve the accuracy of semantic plausibility judgments for simple events expressed in natural language sentences.

Contribution

It introduces a method for injecting detailed entity and event knowledge into LLMs via templates, improving semantic plausibility modeling in realistic scenarios.

Findings

Injected knowledge improves plausibility prediction accuracy

Knowledge augmentation balances label distribution effectively

Identifies importance of non-trivial entity and event types

Abstract

In this work, we investigate the effectiveness of injecting external knowledge to a large language model (LLM) to identify semantic plausibility of simple events. Specifically, we enhance the LLM with fine-grained entity types, event types and their definitions extracted from an external knowledge base. These knowledge are injected into our system via designed templates. We also augment the data to balance the label distribution and adapt the task setting to real world scenarios in which event mentions are expressed as natural language sentences. The experimental results show the effectiveness of the injected knowledge on modeling semantic plausibility of events. An error analysis further emphasizes the importance of identifying non-trivial entity and event types.