Artificial Intelligence can Recognize Whether a Job Applicant is Selling and/or Lying According to Facial Expressions and Head Movements Much More Correctly Than Human Interviewers

TL;DR

This study develops deep learning models using computer vision to detect honesty and deception in job interview videos, outperforming human interviewers in identifying impression management tactics.

Contribution

The paper introduces a novel deep learning approach that accurately recognizes honest and deceptive behaviors from facial cues, surpassing human performance.

Findings

Models explained 91% of variance in honest IMs.

Models explained 84% of variance in deceptive IMs.

Models showed stronger correlations with self-reported IM scores than humans.

Abstract

Whether an interviewee's honest and deceptive responses can be detected by facial expression signals in videos has been debated and requires further research. We developed deep learning models enabled by computer vision to extract temporal patterns of job applicants' facial expressions and head movements to identify self-reported honest and deceptive impression management (IM) tactics from video frames in real asynchronous video interviews. A 12- to 15-minute video was recorded for each of N=121 job applicants as they answered five structured behavioral interview questions. Each applicant completed a survey to self-evaluate their trustworthiness on four IM measures. Additionally, a field experiment was conducted to compare the concurrent validity associated with self-reported IMs between our modeling approach and human interviewers. Human interviewers' performance in predicting these IM measures from another subset of 30 videos was obtained by having N=30 human interviewers evaluate three recordings. Our models explained 91% and 84% of the variance in honest and deceptive IMs, respectively, and showed stronger correlations with self-reported IM scores than human interviewers.