Annotating FrameNet via Structure-Conditioned Language Generation

TL;DR

This paper explores generating semantically structured sentences using language models conditioned on FrameNet, demonstrating high-quality outputs and potential for data augmentation in low-resource scenarios, but with limited benefits in high-resource settings.

Contribution

It introduces a framework for generating frame-semantic annotated sentences conditioned on semantic structures, advancing automatic linguistic annotation techniques.

Findings

Generated sentences are highly accepted by humans.

Semantic conditioning improves low-resource data augmentation.

Limited benefits observed in high-resource settings.

Abstract

Despite the remarkable generative capabilities of language models in producing naturalistic language, their effectiveness on explicit manipulation and generation of linguistic structures remain understudied. In this paper, we investigate the task of generating new sentences preserving a given semantic structure, following the FrameNet formalism. We propose a framework to produce novel frame-semantically annotated sentences following an overgenerate-and-filter approach. Our results show that conditioning on rich, explicit semantic information tends to produce generations with high human acceptance, under both prompting and finetuning. Our generated frame-semantic structured annotations are effective at training data augmentation for frame-semantic role labeling in low-resource settings; however, we do not see benefits under higher resource settings. Our study concludes that while generating high-quality, semantically rich data might be within reach, the downstream utility of such generations remains to be seen, highlighting the outstanding challenges with automating linguistic annotation tasks.