Top1 Solution of QQ Browser 2021 Ai Algorithm Competition Track 1 : Multimodal Video Similarity
Zhuoran Ma, Majing Lou, Xuan Ouyang

TL;DR
This paper presents the winning solution to a video similarity competition using a multi-modal transformer trained with multiple tasks, achieving top leaderboard performance and releasing source code.
Contribution
It introduces a multi-modal transformer approach with multi-task pretraining and fine-tuning for video similarity, winning a major AI competition.
Findings
Achieved 0.852 score on leaderboard
Ensembled multiple models for top performance
Source code released for reproducibility
Abstract
In this paper, we describe the solution to the QQ Browser 2021 Ai Algorithm Competition (AIAC) Track 1. We use the multi-modal transformer model for the video embedding extraction. In the pretrain phase, we train the model with three tasks, (1) Video Tag Classification (VTC), (2) Mask Language Modeling (MLM) and (3) Mask Frame Modeling (MFM). In the finetune phase, we train the model with video similarity based on rank normalized human labels. Our full pipeline, after ensembling several models, scores 0.852 on the leaderboard, which we achieved the 1st place in the competition. The source codes have been released at Github.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Topic Modeling · Human Pose and Action Recognition
