The Good, the Bad, and the Ugly: The Role of AI Quality Disclosure in Lie Detection

TL;DR

This study examines how AI quality disclosures influence lie detection, revealing that low-quality AI without transparency impairs judgment, while high-quality AI consistently improves accuracy, highlighting the importance of transparency in AI-assisted deception detection.

Contribution

The paper demonstrates the impact of AI quality disclosures on lie detection accuracy and shows that transparency can mitigate reliance on low-quality AI advisors.

Findings

Low-quality AI without disclosures reduces truth detection accuracy.

Revealing AI effectiveness restores participants' detection abilities.

High-quality AI improves lie detection regardless of disclosures.

Abstract

We investigate how low-quality AI advisors, lacking quality disclosures, can help spread text-based lies while seeming to help people detect lies. Participants in our experiment discern truth from lies by evaluating transcripts from a game show that mimicked deceptive social media exchanges on topics with objective truths. We find that when relying on low-quality advisors without disclosures, participants' truth-detection rates fall below their own abilities, which recovered once the AI's true effectiveness was revealed. Conversely, high-quality advisor enhances truth detection, regardless of disclosure. We discover that participants' expectations about AI capabilities contribute to their undue reliance on opaque, low-quality advisors.