Detection of Abnormal Behavior with Self-Supervised Gaze Estimation

TL;DR

This paper introduces a self-supervised gaze estimation approach combined with a new dataset and deep learning models to detect abnormal behavior in video conferencing, addressing the urgent need for non-face-to-face interaction solutions during COVID-19.

Contribution

It proposes an optimal gaze estimation network with self-supervised learning, creates a new dataset for anomaly detection, and trains deep models to identify abnormal behaviors in video conferencing.

Findings

Demonstrates improved gaze estimation accuracy with self-supervised learning.

Shows robustness of the anomaly detection method through experiments.

Provides a new dataset for gaze and head pose analysis in video conferencing.

Abstract

Due to the recent outbreak of COVID-19, many classes, exams, and meetings have been conducted non-face-to-face. However, the foundation for video conferencing solutions is still insufficient. So this technology has become an important issue. In particular, these technologies are essential for non-face-to-face testing, and technology dissemination is urgent. In this paper, we present a single video conferencing solution using gaze estimation in preparation for these problems. Gaze is an important cue for the tasks such as analysis of human behavior. Hence, numerous studies have been proposed to solve gaze estimation using deep learning, which is one of the most prominent methods up to date. We use these gaze estimation methods to detect abnormal behavior of video conferencing participants. Our contribution is as follows. i) We find and apply the optimal network for the gaze estimation method and apply a self-supervised method to improve accuracy. ii) For anomaly detection, we present a new dataset that aggregates the values of a new gaze, head pose, etc. iii) We train newly created data on Multi Layer Perceptron (MLP) models to detect anomaly behavior based on deep learning. We demonstrate the robustness of our method through experiments.