WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain

TL;DR

WorkRB is an open-source, community-driven benchmark designed to evaluate AI models across diverse work-related NLP and recommendation tasks, addressing fragmentation and reproducibility issues in employment AI research.

Contribution

It introduces the first comprehensive, modular benchmark tailored to the work domain, enabling standardized evaluation across multiple tasks and multilingual settings.

Findings

Organizes 13 diverse work-related tasks into a unified benchmark.

Supports monolingual and cross-lingual evaluation with multilingual ontologies.

Facilitates contributions and integration of proprietary data through modular design.

Abstract

Today's evolving labor markets rely increasingly on recommender systems for hiring, talent management, and workforce analytics, with natural language processing (NLP) capabilities at the core. Yet, research in this area remains highly fragmented. Studies employ divergent ontologies (ESCO, O*NET, national taxonomies), heterogeneous task formulations, and diverse model families, making cross-study comparison and reproducibility exceedingly difficult. General-purpose benchmarks lack coverage of work-specific tasks, and the inherent sensitivity of employment data further limits open evaluation. We present \textbf{WorkRB} (Work Research Benchmark), the first open-source, community-driven benchmark tailored to work-domain AI. WorkRB organizes 13 diverse tasks from 7 task groups as unified recommendation and NLP tasks, including job/skill recommendation, candidate recommendation, similar item recommendation, and skill extraction and normalization. WorkRB enables both monolingual and cross-lingual evaluation settings through dynamic loading of multilingual ontologies. Developed within a multi-stakeholder ecosystem of academia, industry, and public institutions, WorkRB has a modular design for seamless contributions and enables integration of proprietary tasks without disclosing sensitive data. WorkRB is available under the Apache 2.0 license at https://github.com/techwolf-ai/WorkRB.