Overview of the TalentCLEF 2025: Skill and Job Title Intelligence for Human Capital Management

TL;DR

TalentCLEF 2025 introduces the first public benchmark for skill and job title intelligence, fostering the development of fair, robust, and multilingual language models for human capital management applications.

Contribution

This paper presents TalentCLEF 2025, a novel evaluation campaign with real-world datasets, tasks, and benchmarks for multilingual job title matching and skill prediction.

Findings

Most systems used multilingual encoder models with contrastive learning.

Training strategies impact performance more than model size.

The benchmark attracted 76 teams with over 280 submissions.

Abstract

Advances in natural language processing and large language models are driving a major transformation in Human Capital Management, with a growing interest in building smart systems based on language technologies for talent acquisition, upskilling strategies, and workforce planning. However, the adoption and progress of these technologies critically depend on the development of reliable and fair models, properly evaluated on public data and open benchmarks, which have so far been unavailable in this domain. To address this gap, we present TalentCLEF 2025, the first evaluation campaign focused on skill and job title intelligence. The lab consists of two tasks: Task A - Multilingual Job Title Matching, covering English, Spanish, German, and Chinese; and Task B - Job Title-Based Skill Prediction, in English. Both corpora were built from real job applications, carefully anonymized, and manually annotated to reflect the complexity and diversity of real-world labor market data, including linguistic variability and gender-marked expressions. The evaluations included monolingual and cross-lingual scenarios and covered the evaluation of gender bias. TalentCLEF attracted 76 registered teams with more than 280 submissions. Most systems relied on information retrieval techniques built with multilingual encoder-based models fine-tuned with contrastive learning, and several of them incorporated large language models for data augmentation or re-ranking. The results show that the training strategies have a larger effect than the size of the model alone. TalentCLEF provides the first public benchmark in this field and encourages the development of robust, fair, and transferable language technologies for the labor market.