Egocentric zone-aware action recognition across environments

TL;DR

This paper proposes a method to improve egocentric action recognition by decoupling scene-specific zone appearances from universal zone representations, enhancing cross-domain transferability in diverse environments.

Contribution

It introduces a novel approach to separate domain-specific zone appearance from universal features, boosting transferability of EAR models across different environments.

Findings

Improved cross-domain action recognition accuracy

Effective decoupling of scene-specific and universal zone features

Validated on EPIC-Kitchens-100 and Argo1M datasets

Abstract

Human activities exhibit a strong correlation between actions and the places where these are performed, such as washing something at a sink. More specifically, in daily living environments we may identify particular locations, hereinafter named activity-centric zones, which may afford a set of homogeneous actions. Their knowledge can serve as a prior to favor vision models to recognize human activities. However, the appearance of these zones is scene-specific, limiting the transferability of this prior information to unfamiliar areas and domains. This problem is particularly relevant in egocentric vision, where the environment takes up most of the image, making it even more difficult to separate the action from the context. In this paper, we discuss the importance of decoupling the domain-specific appearance of activity-centric zones from their universal, domain-agnostic representations, and show how the latter can improve the cross-domain transferability of Egocentric Action Recognition (EAR) models. We validate our solution on the EPIC-Kitchens-100 and Argo1M datasets