CUHK & ETHZ & SIAT Submission to ActivityNet Challenge 2016

TL;DR

This paper describes a high-performing video classification method for ActivityNet Challenge 2016, combining advanced deep models, novel aggregation techniques, and audio features, resulting in first place with 93.23% mAP.

Contribution

The paper introduces new aggregation schemes and integrates audio features into the temporal segment network framework for improved untrimmed video classification.

Findings

Achieved 93.23% mAP on the test set

Secured first place in ActivityNet Challenge 2016

Enhanced performance with ensemble of deep models

Abstract

This paper presents the method that underlies our submission to the untrimmed video classification task of ActivityNet Challenge 2016. We follow the basic pipeline of temporal segment networks and further raise the performance via a number of other techniques. Specifically, we use the latest deep model architecture, e.g., ResNet and Inception V3, and introduce new aggregation schemes (top-k and attention-weighted pooling). Additionally, we incorporate the audio as a complementary channel, extracting relevant information via a CNN applied to the spectrograms. With these techniques, we derive an ensemble of deep models, which, together, attains a high classification accuracy (mAP $93.23\%$) on the testing set and secured the first place in the challenge.