# Web-based visualisation of head pose and facial expressions changes: monitoring human activity using depth data

## TL;DR

This paper presents a system that uses affordable 3D sensing technology to monitor human activity by analyzing head pose and facial expressions, enabling visual representations for applications like gaming and HCI.

## Contribution

It introduces a novel platform combining head pose estimation and facial expression recognition using Kinect and discriminative random regression forests.

## Key findings

- Accurate head pose estimation in unconstrained environments.
- Effective recognition of four universal facial expressions.
- JSON-based data exchange for easy data manipulation.

## Abstract

Despite significant recent advances in the field of head pose estimation and facial expression recognition, raising the cognitive level when analysing human activity presents serious challenges to current concepts. Motivated by the need of generating comprehensible visual representations from different sets of data, we introduce a system capable of monitoring human activity through head pose and facial expression changes, utilising an affordable 3D sensing technology (Microsoft Kinect sensor). An approach build on discriminative random regression forests was selected in order to rapidly and accurately estimate head pose changes in unconstrained environment. In order to complete the secondary process of recognising four universal dominant facial expressions (happiness, anger, sadness and surprise), emotion recognition via facial expressions (ERFE) was adopted. After that, a lightweight data exchange format (JavaScript Object Notation-JSON) is employed, in order to manipulate the data extracted from the two aforementioned settings. Such mechanism can yield a platform for objective and effortless assessment of human activity within the context of serious gaming and human-computer interaction.