A Multi-task Joint Framework for Real-time Person Search

TL;DR

This paper introduces a real-time person search framework that jointly optimizes detection, feature extraction, and identity comparison, achieving high accuracy and speed in complex video scenarios.

Contribution

It proposes a multi-task framework with novel modules like YOLOv5-GS, MAA, and 3D pooled table, enhancing detection speed, adaptability, and identification accuracy.

Findings

Detection accuracy improved with YOLOv5-GS.

Framework achieves 93.6% identification rate.

Real-time processing at 25.7 FPS.

Abstract

Person search generally involves three important parts: person detection, feature extraction and identity comparison. However, person search integrating detection, extraction and comparison has the following drawbacks. Firstly, the accuracy of detection will affect the accuracy of comparison. Secondly, it is difficult to achieve real-time in real-world applications. To solve these problems, we propose a Multi-task Joint Framework for real-time person search (MJF), which optimizes the person detection, feature extraction and identity comparison respectively. For the person detection module, we proposed the YOLOv5-GS model, which is trained with person dataset. It combines the advantages of the Ghostnet and the Squeeze-and-Excitation (SE) block, and improves the speed and accuracy. For the feature extraction module, we design the Model Adaptation Architecture (MAA), which could select different network according to the number of people. It could balance the relationship between accuracy and speed. For identity comparison, we propose a Three Dimension (3D) Pooled Table and a matching strategy to improve identification accuracy. On the condition of 1920*1080 resolution video and 500 IDs table, the identification rate (IR) and frames per second (FPS) achieved by our method could reach 93.6% and 25.7,