Facial Descriptors for Human Interaction Recognition In Still Images

TL;DR

This paper introduces a new method for recognizing human interactions in still images by analyzing facial regions, scene features, and deep learning, supported by a newly collected dataset.

Contribution

It proposes a novel facial descriptor-based approach for interaction recognition in still images and introduces a new dataset for this purpose.

Findings

Facial features and scene context are crucial for interaction recognition.

The proposed descriptors improve recognition accuracy.

A new dataset for human interaction in still images was created.

Abstract

This paper presents a novel approach in a rarely studied area of computer vision: Human interaction recognition in still images. We explore whether the facial regions and their spatial configurations contribute to the recognition of interactions. In this respect, our method involves extraction of several visual features from the facial regions, as well as incorporation of scene characteristics and deep features to the recognition. Extracted multiple features are utilized within a discriminative learning framework for recognizing interactions between people. Our designed facial descriptors are based on the observation that relative positions, size and locations of the faces are likely to be important for characterizing human interactions. Since there is no available dataset in this relatively new domain, a comprehensive new dataset which includes several images of human interactions is collected. Our experimental results show that faces and scene characteristics contain important information to recognize interactions between people.