Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework

TL;DR

This paper introduces MSP60K, a large-scale, cross-domain pedestrian attribute dataset, and proposes LLM-PAR, a novel framework augmented with large language models, to improve attribute recognition performance across diverse scenarios.

Contribution

The paper presents a new large-scale, cross-domain pedestrian attribute dataset and a LLM-augmented framework for enhanced recognition, addressing limitations of existing datasets and models.

Findings

MSP60K dataset covers 8 scenarios with 60,122 images and 57 attributes.

LLM-PAR framework improves attribute recognition accuracy across multiple benchmarks.

Synthetic degradation helps bridge the gap between dataset and real-world scenarios.

Abstract

Pedestrian Attribute Recognition (PAR) is one of the indispensable tasks in human-centered research. However, existing datasets neglect different domains (e.g., environments, times, populations, and data sources), only conducting simple random splits, and the performance of these datasets has already approached saturation. In the past five years, no large-scale dataset has been opened to the public. To address this issue, this paper proposes a new large-scale, cross-domain pedestrian attribute recognition dataset to fill the data gap, termed MSP60K. It consists of 60,122 images and 57 attribute annotations across eight scenarios. Synthetic degradation is also conducted to further narrow the gap between the dataset and real-world challenging scenarios. To establish a more rigorous benchmark, we evaluate 17 representative PAR models under both random and cross-domain split protocols on our dataset. Additionally, we propose an innovative Large Language Model (LLM) augmented PAR framework, named LLM-PAR. This framework processes pedestrian images through a Vision Transformer (ViT) backbone to extract features and introduces a multi-embedding query Transformer to learn partial-aware features for attribute classification. Significantly, we enhance this framework with LLM for ensemble learning and visual feature augmentation. Comprehensive experiments across multiple PAR benchmark datasets have thoroughly validated the efficacy of our proposed framework. The dataset and source code accompanying this paper will be made publicly available at \url{https://github.com/Event-AHU/OpenPAR}.