A 3D Face Modelling Approach for Pose-Invariant Face Recognition in a Human-Robot Environment

TL;DR

This paper introduces a real-time 3D face modeling framework that improves pose-invariant face recognition in robotics by transforming non-frontal images into frontal views using landmark-based fitting.

Contribution

The novel contribution is a landmark-based 3D face modeling method that enhances face recognition in in-the-wild robotic environments.

Findings

Improved face recognition accuracy on MUCT and PaSC databases.

Effective pose correction for non-frontal face images.

Integration of the framework into a humanoid robot system.

Abstract

Face analysis techniques have become a crucial component of human-machine interaction in the fields of assistive and humanoid robotics. However, the variations in head-pose that arise naturally in these environments are still a great challenge. In this paper, we present a real-time capable 3D face modelling framework for 2D in-the-wild images that is applicable for robotics. The fitting of the 3D Morphable Model is based exclusively on automatically detected landmarks. After fitting, the face can be corrected in pose and transformed back to a frontal 2D representation that is more suitable for face recognition. We conduct face recognition experiments with non-frontal images from the MUCT database and uncontrolled, in the wild images from the PaSC database, the most challenging face recognition database to date, showing an improved performance. Finally, we present our SCITOS G5 robot system, which incorporates our framework as a means of image pre-processing for face analysis.