Mitigating Language Bias in Cross-Lingual Job Retrieval: A Recruitment Platform Perspective

TL;DR

This paper introduces a unified sentence encoder with a multi-task dual-encoder framework to improve cross-lingual job retrieval by reducing language bias and enhancing overall performance in recruitment platforms.

Contribution

The paper presents a novel unified sentence encoder and a new metric, LBKL, for reducing language bias and improving cross-lingual job matching accuracy.

Findings

Outperforms state-of-the-art models in cross-lingual retrieval tasks.

Significant reduction in language bias as measured by LBKL.

Smaller model size with improved performance.

Abstract

Understanding the textual components of resumes and job postings is critical for improving job-matching accuracy and optimizing job search systems in online recruitment platforms. However, existing works primarily focus on analyzing individual components within this information, requiring multiple specialized tools to analyze each aspect. Such disjointed methods could potentially hinder overall generalizability in recruitment-related text processing. Therefore, we propose a unified sentence encoder that utilized multi-task dual-encoder framework for jointly learning multiple component into the unified sentence encoder. The results show that our method outperforms other state-of-the-art models, despite its smaller model size. Moreover, we propose a novel metric, Language Bias Kullback-Leibler Divergence (LBKL), to evaluate language bias in the encoder, demonstrating significant bias reduction and superior cross-lingual performance.