Real-Time Facial Expression Emoji Masking with Convolutional Neural Networks and Homography

TL;DR

This paper presents a real-time system that detects facial expressions and overlays corresponding emojis on faces using CNNs and homography, suitable for educational environments.

Contribution

The work introduces a real-time facial expression masking system combining face detection, CNN-based emotion classification, and homography for emoji overlay.

Findings

System operates in real-time

Effective in educational settings

Accurate facial expression recognition

Abstract

Neural network based algorithms has shown success in many applications. In image processing, Convolutional Neural Networks (CNN) can be trained to categorize facial expressions of images of human faces. In this work, we create a system that masks a student's face with a emoji of the respective emotion. Our system consists of three building blocks: face detection using Histogram of Gradients (HoG) and Support Vector Machine (SVM), facial expression categorization using CNN trained on FER2013 dataset, and finally masking the respective emoji back onto the student's face via homography estimation. (Demo: https://youtu.be/GCjtXw1y8Pw) Our results show that this pipeline is deploy-able in real-time, and is usable in educational settings.