Bi-modal First Impressions Recognition using Temporally Ordered Deep Audio and Stochastic Visual Features

TL;DR

This paper introduces a bi-modal deep learning approach for first impressions recognition from short videos, leveraging temporally ordered audio and novel stochastic visual features to predict Big Five personality traits with high accuracy.

Contribution

It presents a novel bi-modal neural network architecture that effectively combines audio and visual features for personality trait prediction from limited video data.

Findings

Models perform exceptionally well on the ChaLearn LAP 2016 dataset.

Effective even with small input segments.

Achieved top performance in the APA2016 competition.

Abstract

We propose a novel approach for First Impressions Recognition in terms of the Big Five personality-traits from short videos. The Big Five personality traits is a model to describe human personality using five broad categories: Extraversion, Agreeableness, Conscientiousness, Neuroticism and Openness. We train two bi-modal end-to-end deep neural network architectures using temporally ordered audio and novel stochastic visual features from few frames, without over-fitting. We empirically show that the trained models perform exceptionally well, even after training from a small sub-portions of inputs. Our method is evaluated in ChaLearn LAP 2016 Apparent Personality Analysis (APA) competition using ChaLearn LAP APA2016 dataset and achieved excellent performance.