Scene-Adaptive Person Search via Bilateral Modulations

TL;DR

The paper introduces SEAS, a scene-adaptive person search model that uses bilateral modulations to reduce background and foreground noise, achieving state-of-the-art results across benchmark datasets.

Contribution

SEAS is the first model to apply bilateral modulations for scene noise reduction in person search, enhancing robustness across diverse scenes.

Findings

Achieves 97.1% mAP on CUHK-SYSU dataset.

Achieves 60.5% mAP on PRW dataset.

Outperforms previous state-of-the-art methods.

Abstract

Person search aims to localize specific a target person from a gallery set of images with various scenes. As the scene of moving pedestrian changes, the captured person image inevitably bring in lots of background noise and foreground noise on the person feature, which are completely unrelated to the person identity, leading to severe performance degeneration. To address this issue, we present a Scene-Adaptive Person Search (SEAS) model by introducing bilateral modulations to simultaneously eliminate scene noise and maintain a consistent person representation to adapt to various scenes. In SEAS, a Background Modulation Network (BMN) is designed to encode the feature extracted from the detected bounding box into a multi-granularity embedding, which reduces the input of background noise from multiple levels with norm-aware. Additionally, to mitigate the effect of foreground noise on the person feature, SEAS introduces a Foreground Modulation Network (FMN) to compute the clutter reduction offset for the person embedding based on the feature map of the scene image. By bilateral modulations on both background and foreground within an end-to-end manner, SEAS obtains consistent feature representations without scene noise. SEAS can achieve state-of-the-art (SOTA) performance on two benchmark datasets, CUHK-SYSU with 97.1\% mAP and PRW with 60.5\% mAP. The code is available at https://github.com/whbdmu/SEAS.