whu-nercms at trecvid2021:instance search task

TL;DR

This paper describes the WHU-NERCMS system's participation in TRECVID2021, employing a two-stage retrieval method for person and action detection, fusion strategies, and interactive approaches, achieving top rankings.

Contribution

The paper introduces a novel two-stage retrieval framework combining face recognition and action detection, along with fusion and interaction strategies that outperform previous methods.

Findings

Ranked 1st in both automatic and interactive tracks

Effective fusion of person and action retrieval results

Improved search performance with complementary methods

Abstract

We will make a brief introduction of the experimental methods and results of the WHU-NERCMS in the TRECVID2021 in the paper. This year we participate in the automatic and interactive tasks of Instance Search (INS). For the automatic task, the retrieval target is divided into two parts, person retrieval, and action retrieval. We adopt a two-stage method including face detection and face recognition for person retrieval and two kinds of action detection methods consisting of three frame-based human-object interaction detection methods and two video-based general action detection methods for action retrieval. After that, the person retrieval results and action retrieval results are fused to initialize the result ranking lists. In addition, we make attempts to use complementary methods to further improve search performance. For interactive tasks, we test two different interaction strategies on the fusion results. We submit 4 runs for automatic and interactive tasks respectively. The introduction of each run is shown in Table 1. The official evaluations show that the proposed strategies rank 1st in both automatic and interactive tracks.