VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervision

TL;DR

This paper introduces VLPD, a novel self-supervised approach for pedestrian detection that leverages explicit semantic contexts via vision-language models, improving detection accuracy especially in challenging scenarios.

Contribution

The paper proposes a new self-supervised framework combining vision-language semantic segmentation and contrastive learning for context-aware pedestrian detection without extra annotations.

Findings

Outperforms previous state-of-the-art methods on benchmark datasets.

Effectively detects small-scale and heavily occluded pedestrians.

Utilizes explicit semantic contexts to improve detection robustness.

Abstract

Detecting pedestrians accurately in urban scenes is significant for realistic applications like autonomous driving or video surveillance. However, confusing human-like objects often lead to wrong detections, and small scale or heavily occluded pedestrians are easily missed due to their unusual appearances. To address these challenges, only object regions are inadequate, thus how to fully utilize more explicit and semantic contexts becomes a key problem. Meanwhile, previous context-aware pedestrian detectors either only learn latent contexts with visual clues, or need laborious annotations to obtain explicit and semantic contexts. Therefore, we propose in this paper a novel approach via Vision-Language semantic self-supervision for context-aware Pedestrian Detection (VLPD) to model explicitly semantic contexts without any extra annotations. Firstly, we propose a self-supervised Vision-Language Semantic (VLS) segmentation method, which learns both fully-supervised pedestrian detection and contextual segmentation via self-generated explicit labels of semantic classes by vision-language models. Furthermore, a self-supervised Prototypical Semantic Contrastive (PSC) learning method is proposed to better discriminate pedestrians and other classes, based on more explicit and semantic contexts obtained from VLS. Extensive experiments on popular benchmarks show that our proposed VLPD achieves superior performances over the previous state-of-the-arts, particularly under challenging circumstances like small scale and heavy occlusion. Code is available at https://github.com/lmy98129/VLPD.