Beyond the Resum\'e: A Rubric-Aware Automatic Interview System for Information Elicitation

TL;DR

This paper introduces a system using large language models as interviewers to more effectively elicit nuanced, role-specific information from candidates, aiming to improve early-stage hiring decisions beyond traditional resume screening.

Contribution

It presents a novel LLM-based interview system that updates beliefs about candidate traits in a calibrated manner, enhancing automated candidate evaluation.

Findings

Belief converges towards simulated applicant abilities.

System effectively updates candidate trait beliefs.

Code and data are publicly available.

Abstract

Effective hiring is integral to the success of an organisation, but it is very challenging to find the most suitable candidates because expert evaluation (e.g.\ interviews conducted by a technical manager) are expensive to deploy at scale. Therefore, automated resume scoring and other applicant-screening methods are increasingly used to coarsely filter candidates, making decisions on limited information. We propose that large language models (LLMs) can play the role of subject matter experts to cost-effectively elicit information from each candidate that is nuanced and role-specific, thereby improving the quality of early-stage hiring decisions. We present a system that leverages an LLM interviewer to update belief over an applicant's rubric-oriented latent traits in a calibrated way. We evaluate our system on simulated interviews and show that belief converges towards the simulated applicants' artificially-constructed latent ability levels. We release code, a modest dataset of public-domain/anonymised resumes, belief calibration tests, and simulated interviews, at \href{https://github.com/mbzuai-nlp/beyond-the-resume}{https://github.com/mbzuai-nlp/beyond-the-resume}. Our demo is available at \href{https://btr.hstu.net}{https://btr.hstu.net}.