DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching

TL;DR

This paper presents a DataOps pipeline leveraging AI and machine learning to extract and match labor market skills from resumes and vacancies, integrating diverse data sources for improved societal intelligence and policy support.

Contribution

It introduces a novel DataOps framework for skills extraction and matching, combining data from multiple countries and applying advanced machine learning models.

Findings

Preliminary results demonstrate effective skills extraction from real employment data.

The pipeline successfully matches extracted skills to standard ontologies.

The approach supports policy-making through improved labor market insights.

Abstract

Big Data analytics supported by AI algorithms can support skills localization and retrieval in the context of a labor market intelligence problem. We formulate and solve this problem through specific DataOps models, blending data sources from administrative and technical partners in several countries into cooperation, creating shared knowledge to support policy and decision-making. We then focus on the critical task of skills extraction from resumes and vacancies featuring state-of-the-art machine learning models. We showcase preliminary results with applied machine learning on real data from the employment agencies of the Netherlands and the Flemish region in Belgium. The final goal is to match these skills to standard ontologies of skills, jobs and occupations.