Anticipating human actions by correlating past with the future with Jaccard similarity measures

TL;DR

This paper introduces a novel framework for early human action recognition and anticipation that leverages three new Jaccard-based similarity measures, achieving state-of-the-art results on multiple datasets.

Contribution

The paper presents three innovative Jaccard similarity measures for correlating past and future features, advancing early action recognition and anticipation methods.

Findings

Achieved 91.7% accuracy on UCF101 with 20% observation

Attained 83.5% accuracy on JHMDB with 20% observation

Set new state-of-the-art results on Epic-Kitchen55 and Breakfast datasets

Abstract

We propose a framework for early action recognition and anticipation by correlating past features with the future using three novel similarity measures called Jaccard vector similarity, Jaccard cross-correlation and Jaccard Frobenius inner product over covariances. Using these combinations of novel losses and using our framework, we obtain state-of-the-art results for early action recognition in UCF101 and JHMDB datasets by obtaining 91.7 % and 83.5 % accuracy respectively for an observation percentage of 20. Similarly, we obtain state-of-the-art results for Epic-Kitchen55 and Breakfast datasets for action anticipation by obtaining 20.35 and 41.8 top-1 accuracy respectively.