Three Branches: Detecting Actions With Richer Features

TL;DR

This paper introduces a three-branch model for action recognition that fuses global video, human attention, and long-term activity information, achieving state-of-the-art results in CVPR2019 challenges.

Contribution

The novel three-branch architecture effectively combines multiple levels of information for improved action recognition performance.

Findings

Achieved 21.59% error rate on Kinetics challenge.

Obtained 32.49% mAP on AVA challenge, outperforming previous submissions.

Demonstrated the effectiveness of multi-level feature fusion in action recognition.

Abstract

We present our three branch solutions for International Challenge on Activity Recognition at CVPR2019. This model seeks to fuse richer information of global video clip, short human attention and long-term human activity into a unified model. We have participated in two tasks: Task A, the Kinetics challenge and Task B, spatio-temporal action localization challenge. For Kinetics, we achieve 21.59% error rate. For the AVA challenge, our final model obtains 32.49% mAP on the test sets, which outperforms all submissions to the AVA challenge at CVPR 2018 for more than 10% mAP. As the future work, we will introduce human activity knowledge, which is a new dataset including key information of human activity.