Voting-based Multimodal Automatic Deception Detection

TL;DR

This paper introduces a voting-based multimodal approach for automatic deception detection using video, audio, and text features, achieving high accuracy on two real-world datasets.

Contribution

It proposes a novel ensemble method combining CNN, SVM, and Word2Vec models for multimodal deception detection, outperforming existing techniques.

Findings

Achieved 97% accuracy on image-based deception detection.

Reached 96% accuracy on audio deception detection.

Attained 92% accuracy on text-based deception detection.

Abstract

Automatic Deception Detection has been a hot research topic for a long time, using machine learning and deep learning to automatically detect deception, brings new light to this old field. In this paper, we proposed a voting-based method for automatic deception detection from videos using audio, visual and lexical features. Experiments were done on two datasets, the Real-life trial dataset by Michigan University and the Miami University deception detection dataset. Video samples were split into frames of images, audio, and manuscripts. Our Voting-based Multimodal proposed solution consists of three models. The first model is CNN for detecting deception from images, the second model is Support Vector Machine (SVM) on Mel spectrograms for detecting deception from audio and the third model is Word2Vec on Support Vector Machine (SVM) for detecting deception from manuscripts. Our proposed solution outperforms state of the art. Best results achieved on images, audio and text were 97%, 96%, 92% respectively on Real-Life Trial Dataset, and 97%, 82%, 73% on video, audio and text respectively on Miami University Deception Detection.