Object State Change Classification in Egocentric Videos using the Divided Space-Time Attention Mechanism

TL;DR

This paper introduces a transformer-based model utilizing Divided Space-Time Attention for classifying object state changes in egocentric videos, achieving high performance in the Ego4D challenge and emphasizing the importance of temporal modeling.

Contribution

The paper presents a novel application of Divided Space-Time Attention in a transformer model for egocentric object state change classification, with comprehensive ablation studies.

Findings

Achieved second-best performance in the Ego4D challenge.

Temporal modeling is crucial for accurate object state change classification.

Visualization of model predictions demonstrates interpretability.

Abstract

This report describes our submission called "TarHeels" for the Ego4D: Object State Change Classification Challenge. We use a transformer-based video recognition model and leverage the Divided Space-Time Attention mechanism for classifying object state change in egocentric videos. Our submission achieves the second-best performance in the challenge. Furthermore, we perform an ablation study to show that identifying object state change in egocentric videos requires temporal modeling ability. Lastly, we present several positive and negative examples to visualize our model's predictions. The code is publicly available at: https://github.com/md-mohaiminul/ObjectStateChange