Spontaneous expression classification in the encrypted domain

TL;DR

This paper introduces a novel system for classifying spontaneous facial expressions in encrypted images, enabling privacy-preserving recognition using homomorphic encryption and machine learning techniques.

Contribution

It presents the first system capable of recognizing encrypted spontaneous facial expressions remotely, combining Paillier cryptosystem with Fisher Linear Discriminant Analysis and kNN.

Findings

Successful classification of encrypted spontaneous expressions

First demonstration on NVIE database

Maintains privacy during remote recognition

Abstract

To date, most facial expression analysis have been based on posed image databases and is carried out without being able to protect the identity of the subjects whose expressions are being recognised. In this paper, we propose and implement a system for classifying facial expressions of images in the encrypted domain based on a Paillier cryptosystem implementation of Fisher Linear Discriminant Analysis and k-nearest neighbour (FLDA + kNN). We present results of experiments carried out on a recently developed natural visible and infrared facial expression (NVIE) database of spontaneous images. To the best of our knowledge, this is the first system that will allow the recog-nition of encrypted spontaneous facial expressions by a remote server on behalf of a client.