Improving Keystep Recognition in Ego-Video via Dexterous Focus

TL;DR

This paper introduces a simple yet effective video transformation focusing on stabilized, hand-focused views to improve keystep recognition in egocentric videos, outperforming existing methods without changing model architectures.

Contribution

It proposes a network-architecture-independent video preprocessing technique that enhances keystep recognition in egocentric videos.

Findings

Outperforms existing egocentric video baselines on Ego-Exo4D benchmark

Requires no changes to underlying model infrastructure

Demonstrates the effectiveness of hand-focused video stabilization

Abstract

In this paper, we address the challenge of understanding human activities from an egocentric perspective. Traditional activity recognition techniques face unique challenges in egocentric videos due to the highly dynamic nature of the head during many activities. We propose a framework that seeks to address these challenges in a way that is independent of network architecture by restricting the ego-video input to a stabilized, hand-focused video. We demonstrate that this straightforward video transformation alone outperforms existing egocentric video baselines on the Ego-Exo4D Fine-Grained Keystep Recognition benchmark without requiring any alteration of the underlying model infrastructure.