Multispectral Pedestrian Detection with Sparsely Annotated Label

TL;DR

This paper introduces SAMPD, a novel multispectral pedestrian detection framework that improves pseudo-label quality and diversifies training data in sparsely annotated environments, leading to better detection performance.

Contribution

The paper proposes SAMPD with modules MPAW, PPE, and APRA to enhance pseudo-label quality and diversity in multispectral pedestrian detection under sparse annotations.

Findings

Significant performance improvement over existing methods.

Effective pseudo-label enhancement and diversity augmentation.

Robust detection in sparsely annotated multispectral environments.

Abstract

Although existing Sparsely Annotated Object Detection (SAOD) approches have made progress in handling sparsely annotated environments in multispectral domain, where only some pedestrians are annotated, they still have the following limitations: (i) they lack considerations for improving the quality of pseudo-labels for missing annotations, and (ii) they rely on fixed ground truth annotations, which leads to learning only a limited range of pedestrian visual appearances in the multispectral domain. To address these issues, we propose a novel framework called Sparsely Annotated Multispectral Pedestrian Detection (SAMPD). For limitation (i), we introduce Multispectral Pedestrian-aware Adaptive Weight (MPAW) and Positive Pseudo-label Enhancement (PPE) module. Utilizing multispectral knowledge, these modules ensure the generation of high-quality pseudo-labels and enable effective learning by increasing weights for high-quality pseudo-labels based on modality characteristics. To address limitation (ii), we propose an Adaptive Pedestrian Retrieval Augmentation (APRA) module, which adaptively incorporates pedestrian patches from ground-truth and dynamically integrates high-quality pseudo-labels with the ground-truth, facilitating a more diverse learning pool of pedestrians. Extensive experimental results demonstrate that our SAMPD significantly enhances performance in sparsely annotated environments within the multispectral domain.