Verb Sense Clustering using Contextualized Word Representations for Semantic Frame Induction

TL;DR

This paper investigates how well various contextualized word representations, especially BERT, can recognize and induce semantic frames evoked by verbs in different contexts, advancing understanding of semantic role identification.

Contribution

It compares seven contextualized word representations for semantic frame induction, demonstrating BERT's effectiveness in recognizing verb frames and estimating their number.

Findings

BERT and variants are highly informative for frame induction.

Contextualized representations can estimate the number of frames a verb evokes.

Several representations outperform traditional methods in semantic frame recognition.

Abstract

Contextualized word representations have proven useful for various natural language processing tasks. However, it remains unclear to what extent these representations can cover hand-coded semantic information such as semantic frames, which specify the semantic role of the arguments associated with a predicate. In this paper, we focus on verbs that evoke different frames depending on the context, and we investigate how well contextualized word representations can recognize the difference of frames that the same verb evokes. We also explore which types of representation are suitable for semantic frame induction. In our experiments, we compare seven different contextualized word representations for two English frame-semantic resources, FrameNet and PropBank. We demonstrate that several contextualized word representations, especially BERT and its variants, are considerably informative for semantic frame induction. Furthermore, we examine the extent to which the contextualized representation of a verb can estimate the number of frames that the verb can evoke.