A Global Alignment Kernel based Approach for Group-level Happiness Intensity Estimation

TL;DR

This paper introduces a novel approach using global alignment kernels with support vector regression to estimate group happiness intensity from images, combining facial features and multiple kernel learning for improved accuracy.

Contribution

It proposes a new method integrating GAKs with RVLBP and CNN features, along with a global weight sort scheme and multiple kernel learning strategies for better group happiness analysis.

Findings

Achieves promising results on the HAPPEI database.

Outperforms recent state-of-the-art methods.

Demonstrates effectiveness of combining RVLBP and CNN features.

Abstract

With the progress in automatic human behavior understanding, analysing the perceived affect of multiple people has been recieved interest in affective computing community. Unlike conventional facial expression analysis, this paper primarily focuses on analysing the behaviour of multiple people in an image. The proposed method is based on support vector regression with the combined global alignment kernels (GAKs) to estimate the happiness intensity of a group of people. We first exploit Riesz-based volume local binary pattern (RVLBP) and deep convolutional neural network (CNN) based features for characterizing facial images. Furthermore, we propose to use the GAK for RVLBP and deep CNN features, respectively for explicitly measuring the similarity of two group-level images. Specifically, we exploit the global weight sort scheme to sort the face images from group-level image according to their spatial weights, making an efficient data structure to GAK. Lastly, we propose Multiple kernel learning based on three combination strategies for combining two respective GAKs based on RVLBP and deep CNN features, such that enhancing the discriminative ability of each GAK. Intensive experiments are performed on the challenging group-level happiness intensity database, namely HAPPEI. Our experimental results demonstrate that the proposed approach achieves promising performance for group happiness intensity analysis, when compared with the recent state-of-the-art methods.