Attacks against Ranking Algorithms with Text Embeddings: a Case Study on Recruitment Algorithms
Anahita Samadi, Debapriya Banerjee, Shirin Nilizadeh

TL;DR
This paper investigates vulnerabilities in recruitment ranking algorithms that use text embeddings, demonstrating successful white and black box attacks that can manipulate resume rankings, with TF IDF-based methods being more susceptible.
Contribution
It is the first to analyze and demonstrate attack strategies against ranking algorithms utilizing text embeddings in recruitment contexts.
Findings
Attacks successfully increase resume rankings in both white and black box scenarios.
TF IDF-based ranking algorithms are more vulnerable to attacks than USE-based methods.
Adversaries can significantly manipulate ranking outcomes using identified influential text items.
Abstract
Recently, some studies have shown that text classification tasks are vulnerable to poisoning and evasion attacks. However, little work has investigated attacks against decision making algorithms that use text embeddings, and their output is a ranking. In this paper, we focus on ranking algorithms for recruitment process, that employ text embeddings for ranking applicants resumes when compared to a job description. We demonstrate both white box and black box attacks that identify text items, that based on their location in embedding space, have significant contribution in increasing the similarity score between a resume and a job description. The adversary then uses these text items to improve the ranking of their resume among others. We tested recruitment algorithms that use the similarity scores obtained from Universal Sentence Encoder (USE) and Term Frequency Inverse Document…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsMultilingual Universal Sentence Encoder
