Transformer-Based Person Search with High-Frequency Augmentation and Multi-Wave Mixing

TL;DR

This paper introduces HAMW, a transformer-based person search method that enhances high-frequency feature perception and reduces computational costs through multi-wave mixing and wavelet fusion, achieving state-of-the-art results.

Contribution

The paper proposes a novel HAMW framework that improves high-frequency feature extraction and efficiency in transformer-based person search models using multi-wave mixing and wavelet fusion.

Findings

HAMW outperforms existing methods on CUHK-SYSU and PRW datasets.

The multi-wave mixing enhances high-frequency feature learning.

Wavelet fusion reduces computational complexity.

Abstract

The person search task aims to locate a target person within a set of scene images. In recent years, transformer-based models in this field have made some progress. However, they still face three primary challenges: 1) the self-attention mechanism tends to suppress high-frequency components in the features, which severely impacts model performance; 2) the computational cost of transformers is relatively high. To address these issues, we propose a novel High-frequency Augmentation and Multi-Wave mixing (HAMW) method for person search. HAMW is designed to enhance the discriminative feature extraction capabilities of transformers while reducing computational overhead and improving efficiency. Specifically, we develop a three-stage framework that progressively optimizes both detection and re-identification performance. Our model enhances the perception of high-frequency features by learning from augmented inputs containing additional high-frequency components. Furthermore, we replace the self-attention layers in the transformer with a strategy based on multi-level Haar wavelet fusion to capture multi-scale features. This not only lowers the computational complexity but also alleviates the suppression of high-frequency features and enhances the ability to exploit multi-scale information. Extensive experiments demonstrate that HAMW achieves state-of-the-art performance on both the CUHK-SYSU and PRW datasets.