Mining Logical Event Schemas From Pre-Trained Language Models

TL;DR

This paper introduces NESL, a system that learns complex event schemas from language models using FrameNet parsing and behavioral schemas, enabling more comprehensive understanding of everyday scenarios without pre-existing story corpora.

Contribution

The paper presents NESL, a novel framework combining language models, FrameNet, and logical schemas to learn hierarchical event schemas from continuous language data.

Findings

Schemas specify situations more comprehensively than other systems.

Careful sampling emphasizes stereotypical properties and reduces irrelevant details.

Hierarchical schemas capture complex everyday scenarios.

Abstract

We present NESL (the Neuro-Episodic Schema Learner), an event schema learning system that combines large language models, FrameNet parsing, a powerful logical representation of language, and a set of simple behavioral schemas meant to bootstrap the learning process. In lieu of a pre-made corpus of stories, our dataset is a continuous feed of "situation samples" from a pre-trained language model, which are then parsed into FrameNet frames, mapped into simple behavioral schemas, and combined and generalized into complex, hierarchical schemas for a variety of everyday scenarios. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details, and that the resulting schemas specify situations more comprehensively than those learned by other systems.