LoRA-like Calibration for Multimodal Deception Detection using ATSFace Data

TL;DR

This paper presents an attention-aware multimodal neural network for deception detection that combines a LoRA-inspired calibration method with ATSFace data, achieving high accuracy and interpretability in identifying deceptive cues.

Contribution

Introduces a novel attention-aware multimodal deception detection model with a LoRA-inspired calibration technique and a new dataset, ATSFace, enhancing accuracy and interpretability.

Findings

92% accuracy on ATSFace dataset

Effective identification of deception cues through attention focus

Calibration improves individual detection accuracy

Abstract

Recently, deception detection on human videos is an eye-catching techniques and can serve lots applications. AI model in this domain demonstrates the high accuracy, but AI tends to be a non-interpretable black box. We introduce an attention-aware neural network addressing challenges inherent in video data and deception dynamics. This model, through its continuous assessment of visual, audio, and text features, pinpoints deceptive cues. We employ a multimodal fusion strategy that enhances accuracy; our approach yields a 92\% accuracy rate on a real-life trial dataset. Most important of all, the model indicates the attention focus in the videos, providing valuable insights on deception cues. Hence, our method adeptly detects deceit and elucidates the underlying process. We further enriched our study with an experiment involving students answering questions either truthfully or deceitfully, resulting in a new dataset of 309 video clips, named ATSFace. Using this, we also introduced a calibration method, which is inspired by Low-Rank Adaptation (LoRA), to refine individual-based deception detection accuracy.