Feature Learning from Spectrograms for Assessment of Personality Traits

TL;DR

This paper introduces a spectrogram-based feature learning method for personality trait assessment that simplifies feature extraction while achieving state-of-the-art accuracy, reducing complexity and parameter tuning.

Contribution

It proposes a novel spectrogram patch-based feature learning approach that simplifies and improves personality trait classification from speech.

Findings

Achieves state-of-the-art classification accuracy.

Reduces feature extraction complexity and parameter tuning.

Uses a single descriptor type for automatic parameter tuning.

Abstract

Several methods have recently been proposed to analyze speech and automatically infer the personality of the speaker. These methods often rely on prosodic and other hand crafted speech processing features extracted with off-the-shelf toolboxes. To achieve high accuracy, numerous features are typically extracted using complex and highly parameterized algorithms. In this paper, a new method based on feature learning and spectrogram analysis is proposed to simplify the feature extraction process while maintaining a high level of accuracy. The proposed method learns a dictionary of discriminant features from patches extracted in the spectrogram representations of training speech segments. Each speech segment is then encoded using the dictionary, and the resulting feature set is used to perform classification of personality traits. Experiments indicate that the proposed method achieves state-of-the-art results with a significant reduction in complexity when compared to the most recent reference methods. The number of features, and difficulties linked to the feature extraction process are greatly reduced as only one type of descriptors is used, for which the 6 parameters can be tuned automatically. In contrast, the simplest reference method uses 4 types of descriptors to which 6 functionals are applied, resulting in over 20 parameters to be tuned.