SAIC_Cambridge-HuPBA-FBK Submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021

TL;DR

This paper details a submission to the EPIC-Kitchens-100 challenge using novel spatio-temporal models GSF and XViT, achieving 44.82% accuracy with RGB inputs.

Contribution

It introduces GSF and XViT models for efficient video action recognition and demonstrates their ensemble effectiveness in a competitive challenge.

Findings

Achieved 44.82% top-1 accuracy on the EPIC-Kitchens-100 leaderboard.

Developed GSF as an efficient spatio-temporal feature extractor for 2D CNNs.

Designed XViT as a transformer-based, convolution-free video feature extractor.

Abstract

This report presents the technical details of our submission to the EPIC-Kitchens-100 Action Recognition Challenge 2021. To participate in the challenge we deployed spatio-temporal feature extraction and aggregation models we have developed recently: GSF and XViT. GSF is an efficient spatio-temporal feature extracting module that can be plugged into 2D CNNs for video action recognition. XViT is a convolution free video feature extractor based on transformer architecture. We design an ensemble of GSF and XViT model families with different backbones and pretraining to generate the prediction scores. Our submission, visible on the public leaderboard, achieved a top-1 action recognition accuracy of 44.82%, using only RGB.