Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies

TL;DR

This study compares encoder-based NER models and large language models for extracting skills from Russian job vacancies, finding traditional NER models outperform LLMs in accuracy and efficiency, thus improving skill identification in the labor market.

Contribution

It provides a comparative analysis of traditional NER and LLM approaches for skill extraction in Russian job descriptions, highlighting the superior performance of NER models in this context.

Findings

Traditional NER models outperform LLMs in accuracy and inference time.

DeepPavlov RuBERT NER tuned achieves the best results among NER models.

NER models are more effective for skill extraction in Russian job vacancies.

Abstract

The labor market is undergoing rapid changes, with increasing demands on job seekers and a surge in job openings. Identifying essential skills and competencies from job descriptions is challenging due to varying employer requirements and the omission of key skills. This study addresses these challenges by comparing traditional Named Entity Recognition (NER) methods based on encoders with Large Language Models (LLMs) for extracting skills from Russian job vacancies. Using a labeled dataset of 4,000 job vacancies for training and 1,472 for testing, the performance of both approaches is evaluated. Results indicate that traditional NER models, especially DeepPavlov RuBERT NER tuned, outperform LLMs across various metrics including accuracy, precision, recall, and inference time. The findings suggest that traditional NER models provide more effective and efficient solutions for skill extraction, enhancing job requirement clarity and aiding job seekers in aligning their qualifications with employer expectations. This research contributes to the field of natural language processing (NLP) and its application in the labor market, particularly in non-English contexts.