One-stage Action Detection Transformer

TL;DR

This paper presents a novel one-stage transformer-based approach for action detection in videos, achieving top performance in the EPIC-KITCHENS-100 challenge by modeling temporal connections and recognizing categories and boundaries simultaneously.

Contribution

The introduction of the One-stage Action Detection Transformer (OADT) that models temporal connections and performs joint recognition of categories and boundaries.

Findings

Achieved 21.28% action mAP on the challenge test set.

Ranked 1st in the EPIC-KITCHENS-100 2022 Action Detection challenge.

Ensembling multiple models improved detection performance.

Abstract

In this work, we introduce our solution to the EPIC-KITCHENS-100 2022 Action Detection challenge. One-stage Action Detection Transformer (OADT) is proposed to model the temporal connection of video segments. With the help of OADT, both the category and time boundary can be recognized simultaneously. After ensembling multiple OADT models trained from different features, our model can reach 21.28\% action mAP and ranks the 1st on the test-set of the Action detection challenge.