# Deep Dual Relation Modeling for Egocentric Interaction Recognition

## TL;DR

This paper introduces a dual relation modeling framework with an interactive LSTM to explicitly capture relations between the camera wearer and interactor, improving egocentric interaction recognition accuracy.

## Contribution

The paper proposes a novel interactive LSTM module and a dual relation modeling framework that explicitly models relations between interacting persons in egocentric videos.

## Key findings

- Outperforms state-of-the-art methods on three datasets.
- Effectively models relations between camera wearer and interactor.
- Demonstrates the importance of relation modeling in egocentric interaction recognition.

## Abstract

Egocentric interaction recognition aims to recognize the camera wearer's interactions with the interactor who faces the camera wearer in egocentric videos. In such a human-human interaction analysis problem, it is crucial to explore the relations between the camera wearer and the interactor. However, most existing works directly model the interactions as a whole and lack modeling the relations between the two interacting persons. To exploit the strong relations for egocentric interaction recognition, we introduce a dual relation modeling framework which learns to model the relations between the camera wearer and the interactor based on the individual action representations of the two persons. Specifically, we develop a novel interactive LSTM module, the key component of our framework, to explicitly model the relations between the two interacting persons based on their individual action representations, which are collaboratively learned with an interactor attention module and a global-local motion module. Experimental results on three egocentric interaction datasets show the effectiveness of our method and advantage over state-of-the-arts.