AutoScreen-FW: An LLM-based Framework for Resume Screening

TL;DR

AutoScreen-FW is a novel open-source framework that enables efficient, privacy-preserving resume screening using locally run LLMs, outperforming some commercial models in judgment accuracy and significantly reducing screening time.

Contribution

The paper introduces AutoScreen-FW, a new framework for resume screening that selects representative samples for in-context learning with open-source LLMs, improving judgment performance without relying on commercial models.

Findings

AutoScreen-FW outperforms GPT-5-nano in judgment accuracy.

It surpasses GPT-5-mini under certain conditions.

It runs substantially faster than commercial GPT models.

Abstract

Corporate recruiters often need to screen many resumes within a limited time, which increases their burden and may cause suitable candidates to be overlooked. To address these challenges, prior work has explored LLM-based automated resume screening. However, some methods rely on commercial LLMs, which may pose data privacy risks. Moreover, since companies typically do not make resumes with evaluation results publicly available, it remains unclear which resume samples should be used during learning to improve an LLM's judgment performance. To address these problems, we propose AutoScreen-FW, an LLM-based locally and automatically resume screening framework. AutoScreen-FW uses several methods to select a small set of representative resume samples. These samples are used for in-context learning together with a persona description and evaluation criteria, enabling open-source LLMs to act as a career advisor and evaluate unseen resumes. Experiments with multiple ground truths show that the open-source LLM judges consistently outperform GPT-5-nano. Under one ground truth setting, it also surpass GPT-5-mini. Although it is slightly weaker than GPT-5-mini under other ground-truth settings, it runs substantially faster per resume than commercial GPT models. These findings indicate the potential for deploying AutoScreen-FW locally in companies to support efficient screening while reducing recruiters' burden.