Face-space Action Recognition by Face-Object Interactions

TL;DR

This paper introduces a novel face-space action recognition method that explicitly models face-related actions by integrating human face, object, and interaction features, achieving significant improvements over previous methods.

Contribution

It presents a new approach focusing on face-related actions, using class-specific features and relations, with a novel two-cycle process for improved recognition accuracy.

Findings

52% average relative improvement over state-of-the-art methods

Developed a new benchmark for face-related action recognition

Demonstrated effectiveness on challenging face-related actions

Abstract

Action recognition in still images has seen major improvement in recent years due to advances in human pose estimation, object recognition and stronger feature representations. However, there are still many cases in which performance remains far from that of humans. In this paper, we approach the problem by learning explicitly, and then integrating three components of transitive actions: (1) the human body part relevant to the action (2) the object being acted upon and (3) the specific form of interaction between the person and the object. The process uses class-specific features and relations not used in the past for action recognition and which use inherently two cycles in the process unlike most standard approaches. We focus on face-related actions (FRA), a subset of actions that includes several currently challenging categories. We present an average relative improvement of 52% over state-of-the art. We also make a new benchmark publicly available.