Job-SDF: A Multi-Granularity Dataset for Job Skill Demand Forecasting and Benchmarking

TL;DR

This paper introduces Job-SDF, a comprehensive dataset derived from over 10 million job ads in China, enabling multi-level skill demand forecasting and benchmarking models to improve workforce planning.

Contribution

It provides the first large-scale, multi-granularity dataset for job skill demand forecasting and benchmarks various models, offering new insights into their performance under different scenarios.

Findings

Benchmarking results reveal model strengths and weaknesses.

Dataset enables evaluation at occupation, company, and regional levels.

Insights into model performance during structural breaks.

Abstract

In a rapidly evolving job market, skill demand forecasting is crucial as it enables policymakers and businesses to anticipate and adapt to changes, ensuring that workforce skills align with market needs, thereby enhancing productivity and competitiveness. Additionally, by identifying emerging skill requirements, it directs individuals towards relevant training and education opportunities, promoting continuous self-learning and development. However, the absence of comprehensive datasets presents a significant challenge, impeding research and the advancement of this field. To bridge this gap, we present Job-SDF, a dataset designed to train and benchmark job-skill demand forecasting models. Based on 10.35 million public job advertisements collected from major online recruitment platforms in China between 2021 and 2023, this dataset encompasses monthly recruitment demand for 2,324 types of skills across 521 companies. Our dataset uniquely enables evaluating skill demand forecasting models at various granularities, including occupation, company, and regional levels. We benchmark a range of models on this dataset, evaluating their performance in standard scenarios, in predictions focused on lower value ranges, and in the presence of structural breaks, providing new insights for further research. Our code and dataset are publicly accessible via the https://github.com/Job-SDF/benchmark.