Prompt2Gaussia: Uncertain Prompt-learning for Script Event Prediction

TL;DR

This paper introduces Prompt2Gaussia, a novel approach for Script Event Prediction that models prompt and label uncertainties using Gaussian distributions within pre-trained language models, improving prediction accuracy.

Contribution

It pioneers the application of Gaussian-based uncertainty modeling in prompt-learning for script event prediction, addressing scenario-diversity and label-ambiguity issues.

Findings

Outperforms prior baselines by 1.46% and 1.05% on two benchmarks.

Effectively models prompt and label uncertainties with Gaussian distributions.

Demonstrates the viability of prompt-learning in script event prediction.

Abstract

Script Event Prediction (SEP) aims to predict the subsequent event for a given event chain from a candidate list. Prior research has achieved great success by integrating external knowledge to enhance the semantics, but it is laborious to acquisite the appropriate knowledge resources and retrieve the script-related knowledge. In this paper, we regard public pre-trained language models as knowledge bases and automatically mine the script-related knowledge via prompt-learning. Still, the scenario-diversity and label-ambiguity in scripts make it uncertain to construct the most functional prompt and label token in prompt learning, i.e., prompt-uncertainty and verbalizer-uncertainty. Considering the innate ability of Gaussian distribution to express uncertainty, we deploy the prompt tokens and label tokens as random variables following Gaussian distributions, where a prompt estimator and a verbalizer estimator are proposed to estimate their probabilistic representations instead of deterministic representations. We take the lead to explore prompt-learning in SEP and provide a fresh perspective to enrich the script semantics. Our method is evaluated on the most widely used benchmark and a newly proposed large-scale one. Experiments show that our method, which benefits from knowledge evoked from pre-trained language models, outperforms prior baselines by 1.46\% and 1.05\% on two benchmarks, respectively.