Rethinking Skill Extraction in the Job Market Domain using Large Language Models

TL;DR

This paper investigates using large language models with in-context learning to improve skill extraction from job-related documents, addressing limitations of traditional supervised methods and better capturing complex skill mentions.

Contribution

It introduces a novel approach leveraging LLMs for skill extraction, demonstrating improved handling of complex skill patterns over traditional supervised models.

Findings

LLMs can better handle syntactically complex skill mentions.

In-context learning with LLMs offers a promising alternative to supervised models.

Performance of LLMs is comparable but not superior to traditional methods.

Abstract

Skill Extraction involves identifying skills and qualifications mentioned in documents such as job postings and resumes. The task is commonly tackled by training supervised models using a sequence labeling approach with BIO tags. However, the reliance on manually annotated data limits the generalizability of such approaches. Moreover, the common BIO setting limits the ability of the models to capture complex skill patterns and handle ambiguous mentions. In this paper, we explore the use of in-context learning to overcome these challenges, on a benchmark of 6 uniformized skill extraction datasets. Our approach leverages the few-shot learning capabilities of large language models (LLMs) to identify and extract skills from sentences. We show that LLMs, despite not being on par with traditional supervised models in terms of performance, can better handle syntactically complex skill mentions in skill extraction tasks.