Exploring In-Context Learning for Frame-Semantic Parsing

TL;DR

This paper explores using in-context learning with large language models for frame-semantic parsing, demonstrating that prompt-based methods can achieve competitive results without fine-tuning on domain-specific tasks.

Contribution

It introduces an automatic prompt generation approach for frame identification and semantic role labeling, enabling LLMs to perform FSP effectively without model fine-tuning.

Findings

Achieved F1 scores of 94.3% for frame identification and 77.4% for semantic role labeling.

ICL with prompts provides a practical alternative to fine-tuning for domain-specific FSP.

Method is effective on a subset of frames related to violent events.

Abstract

Frame Semantic Parsing (FSP) entails identifying predicates and labeling their arguments according to Frame Semantics. This paper investigates the use of In-Context Learning (ICL) with Large Language Models (LLMs) to perform FSP without model fine-tuning. We propose a method that automatically generates task-specific prompts for the Frame Identification (FI) and Frame Semantic Role Labeling (FSRL) subtasks, relying solely on the FrameNet database. These prompts, constructed from frame definitions and annotated examples, are used to guide six different LLMs. Experiments are conducted on a subset of frames related to violent events. The method achieves competitive results, with F1 scores of 94.3% for FI and 77.4% for FSRL. The findings suggest that ICL offers a practical and effective alternative to traditional fine-tuning for domain-specific FSP tasks.