Variational Gaussian Process Auto-Encoder for Ordinal Prediction of Facial Action Units

TL;DR

This paper introduces a variational Gaussian process auto-encoder that effectively fuses features and models ordinal outputs, specifically applied to facial action unit prediction, demonstrating superior performance on benchmark datasets.

Contribution

It presents a novel GP auto-encoder framework that integrates ordinal constraints into the latent space for improved feature fusion and ordinal prediction.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effectively models ordinal structure in facial action unit prediction.

Provides robust feature fusion for affect analysis.

Abstract

We address the task of simultaneous feature fusion and modeling of discrete ordinal outputs. We propose a novel Gaussian process(GP) auto-encoder modeling approach. In particular, we introduce GP encoders to project multiple observed features onto a latent space, while GP decoders are responsible for reconstructing the original features. Inference is performed in a novel variational framework, where the recovered latent representations are further constrained by the ordinal output labels. In this way, we seamlessly integrate the ordinal structure in the learned manifold, while attaining robust fusion of the input features. We demonstrate the representation abilities of our model on benchmark datasets from machine learning and affect analysis. We further evaluate the model on the tasks of feature fusion and joint ordinal prediction of facial action units. Our experiments demonstrate the benefits of the proposed approach compared to the state of the art.