An Empirical Study of Mamba-based Pedestrian Attribute Recognition

TL;DR

This study evaluates the effectiveness of Mamba-based architectures for pedestrian attribute recognition, comparing them with Transformer models and exploring hybrid designs to optimize accuracy and computational efficiency.

Contribution

It adapts Mamba into pedestrian attribute recognition frameworks, investigates hybrid Mamba-Transformer models, and provides comprehensive experimental validation of their performance.

Findings

Interacting with attribute tags does not always improve performance.

Hybrid Mamba-Transformer models can outperform pure Mamba models under certain conditions.

Simply adding Transformer components to Mamba does not guarantee better results.

Abstract

Current strong pedestrian attribute recognition models are developed based on Transformer networks, which are computationally heavy. Recently proposed models with linear complexity (e.g., Mamba) have garnered significant attention and have achieved a good balance between accuracy and computational cost across a variety of visual tasks. Relevant review articles also suggest that while these models can perform well on some pedestrian attribute recognition datasets, they are generally weaker than the corresponding Transformer models. To further tap into the potential of the novel Mamba architecture for PAR tasks, this paper designs and adapts Mamba into two typical PAR frameworks, i.e., the text-image fusion approach and pure vision Mamba multi-label recognition framework. It is found that interacting with attribute tags as additional input does not always lead to an improvement, specifically, Vim can be enhanced, but VMamba cannot. This paper further designs various hybrid Mamba-Transformer variants and conducts thorough experimental validations. These experimental results indicate that simply enhancing Mamba with a Transformer does not always lead to performance improvements but yields better results under certain settings. We hope this empirical study can further inspire research in Mamba for PAR, and even extend into the domain of multi-label recognition, through the design of these network structures and comprehensive experimentation. The source code of this work will be released at \url{https://github.com/Event-AHU/OpenPAR}