When lies are mostly truthful: automated verbal deception detection for embedded lies

TL;DR

This paper introduces a new dataset and demonstrates that a fine-tuned language model can detect embedded lies within truthful statements with 64% accuracy, highlighting the complexity of verbal deception detection.

Contribution

The study provides a novel dataset of embedded lies and shows that current language models can classify deceptive statements with moderate accuracy, advancing research in realistic deception detection.

Findings

Language model achieves 64% accuracy in detecting embedded lies.

Embedded lies closely resemble truthful statements linguistically.

Most deceptive statements contain about 1/3 embedded lies.

Abstract

Background: Verbal deception detection research relies on narratives and commonly assumes statements as truthful or deceptive. A more realistic perspective acknowledges that the veracity of statements exists on a continuum with truthful and deceptive parts being embedded within the same statement. However, research on embedded lies has been lagging behind. Methods: We collected a novel dataset of 2,088 truthful and deceptive statements with annotated embedded lies. Using a within-subjects design, participants provided a truthful account of an autobiographical event. They then rewrote their statement in a deceptive manner by including embedded lies, which they highlighted afterwards and judged on lie centrality, deceptiveness, and source. Results: We show that a fined-tuned language model (Llama-3-8B) can classify truthful statements and those containing embedded lies with 64% accuracy. Individual differences, linguistic properties and explainability analysis suggest that the challenge of moving the dial towards embedded lies stems from their resemblance to truthful statements. Typical deceptive statements consisted of 2/3 truthful information and 1/3 embedded lies, largely derived from past personal experiences and with minimal linguistic differences with their truthful counterparts. Conclusion: We present this dataset as a novel resource to address this challenge and foster research on embedded lies in verbal deception detection.