Leveraging Self-Supervised Training for Unintentional Action Recognition

TL;DR

This paper introduces a multi-stage, self-supervised framework for recognizing unintentional actions in videos by exploiting inherent biases and temporal transformations, significantly advancing the state-of-the-art.

Contribution

It proposes a novel multi-stage approach combined with self-supervised temporal transformations (T2IBUA) to improve unintentional action recognition in videos.

Findings

Strong performance improvements over existing methods

Effective modeling of temporal information at multiple levels

Extensive ablation studies validate the framework's effectiveness

Abstract

Unintentional actions are rare occurrences that are difficult to define precisely and that are highly dependent on the temporal context of the action. In this work, we explore such actions and seek to identify the points in videos where the actions transition from intentional to unintentional. We propose a multi-stage framework that exploits inherent biases such as motion speed, motion direction, and order to recognize unintentional actions. To enhance representations via self-supervised training for the task of unintentional action recognition we propose temporal transformations, called Temporal Transformations of Inherent Biases of Unintentional Actions (T2IBUA). The multi-stage approach models the temporal information on both the level of individual frames and full clips. These enhanced representations show strong performance for unintentional action recognition tasks. We provide an extensive ablation study of our framework and report results that significantly improve over the state-of-the-art.