InFillmore: Frame-Guided Language Generation with Bidirectional Context

TL;DR

InFillmore introduces a frame-guided language generation method that uses bidirectional context and frame semantics to control infilling, demonstrated through fine-tuning and constrained decoding, with promising evaluation results.

Contribution

The paper presents a novel approach combining frame semantics with bidirectional infilling, including a new decoding extension and fine-tuning strategies for explicit semantic control.

Findings

Frame-guided generation enables explicit semantic manipulation.

Methods maintain high text distinguishability from human writing.

Approach is flexible across various use scenarios.

Abstract

We propose a structured extension to bidirectional-context conditional language generation, or "infilling," inspired by Frame Semantic theory (Fillmore, 1976). Guidance is provided through two approaches: (1) model fine-tuning, conditioning directly on observed symbolic frames, and (2) a novel extension to disjunctive lexically constrained decoding that leverages frame semantic lexical units. Automatic and human evaluations confirm that frame-guided generation allows for explicit manipulation of intended infill semantics, with minimal loss in distinguishability from human-generated text. Our methods flexibly apply to a variety of use scenarios, and we provide a codebase and interactive demo available from https://nlp.jhu.edu/demos/infillmore.