Enhancing PLM Performance on Labour Market Tasks via Instruction-based Finetuning and Prompt-tuning with Rules

TL;DR

This paper explores prompt-based tuning of pre-trained language models to improve performance on labour market tasks, emphasizing cost-effective methods that do not require extensive annotations or model modifications.

Contribution

It demonstrates that instruction tuning and prompt-tuning without exemplars enhance PLM performance on labour market applications efficiently.

Findings

Prompt-tuning improves task-specific accuracy.

Instruction tuning without exemplars is effective.

Methods reduce reliance on annotated data.

Abstract

The increased digitization of the labour market has given researchers, educators, and companies the means to analyze and better understand the labour market. However, labour market resources, although available in high volumes, tend to be unstructured, and as such, research towards methodologies for the identification, linking, and extraction of entities becomes more and more important. Against the backdrop of this quest for better labour market representations, resource constraints and the unavailability of large-scale annotated data cause a reliance on human domain experts. We demonstrate the effectiveness of prompt-based tuning of pre-trained language models (PLM) in labour market specific applications. Our results indicate that cost-efficient methods such as PTR and instruction tuning without exemplars can significantly increase the performance of PLMs on downstream labour market applications without introducing additional model layers, manual annotations, and data augmentation.