Multi-view Laplacian Eigenmaps Based on Bag-of-Neighbors For RGBD Human Emotion Recognition

TL;DR

This paper introduces a novel multi-view Laplacian Eigenmaps method combined with a neural network for RGBD human emotion recognition, utilizing new datasets and a unique distance metric to improve feature representation and classification accuracy.

Contribution

It presents the first RGBD video-emotion dataset and a new supervised multi-view Laplacian Eigenmaps approach with a multihidden-layer network for emotion recognition.

Findings

Effective in handling different feature sizes and sample numbers.

Outperforms some state-of-the-art methods.

Extensible to more than two views.

Abstract

Human emotion recognition is an important direction in the field of biometric and information forensics. However, most existing human emotion research are based on the single RGB view. In this paper, we introduce a RGBD video-emotion dataset and a RGBD face-emotion dataset for research. To our best knowledge, this may be the first RGBD video-emotion dataset. We propose a new supervised nonlinear multi-view laplacian eigenmaps (MvLE) approach and a multihidden-layer out-of-sample network (MHON) for RGB-D humanemotion recognition. To get better representations of RGB view and depth view, MvLE is used to map the training set of both views from original space into the common subspace. As RGB view and depth view lie in different spaces, a new distance metric bag of neighbors (BON) used in MvLE can get the similar distributions of the two views. Finally, MHON is used to get the low-dimensional representations of test data and predict their labels. MvLE can deal with the cases that RGB view and depth view have different size of features, even different number of samples and classes. And our methods can be easily extended to more than two views. The experiment results indicate the effectiveness of our methods over some state-of-art methods.