SkillNER: Mining and Mapping Soft Skills from any Text

TL;DR

SkillNER is a novel data-driven named entity recognition system that automatically extracts soft skills from text, aiding HR and psychological research by enabling large-scale analysis of soft skills in various textual sources.

Contribution

It introduces SkillNER, a support vector machine-based NER system trained on scientific papers, validated with psychologists, and applied to real-world job descriptions for soft skills extraction.

Findings

Successfully extracted soft skills from 5000+ scientific papers

Enabled detection of job profile communities based on shared soft skills

Demonstrated efficiency and practical utility in real-world case studies

Abstract

In today's digital world, there is an increasing focus on soft skills. On the one hand, they facilitate innovation at companies, but on the other, they are unlikely to be automated soon. Researchers struggle with accurately approaching quantitatively the study of soft skills due to the lack of data-driven methods to retrieve them. This limits the possibility for psychologists and HR managers to understand the relation between humans and digitalisation. This paper presents SkillNER, a novel data-driven method for automatically extracting soft skills from text. It is a named entity recognition (NER) system trained with a support vector machine (SVM) on a corpus of more than 5000 scientific papers. We developed this system by measuring the performance of our approach against different training models and validating the results together with a team of psychologists. Finally, SkillNER was tested in a real-world case study using the job descriptions of ESCO (European Skill/Competence Qualification and Occupation) as textual source. The system enabled the detection of communities of job profiles based on their shared soft skills and communities of soft skills based on their shared job profiles. This case study demonstrates that the tool can automatically retrieve soft skills from a large corpus in an efficient way, proving useful for firms, institutions, and workers. The tool is open and available online to foster quantitative methods for the study of soft skills.