Color inference from semantic labeling for person search in videos

TL;DR

This paper introduces an explainable, semantic color labeling method for person search in videos, improving color label accuracy by modeling human perception and handling variability, leading to high-precision pedestrian labeling.

Contribution

The work presents a novel color inference approach using binary search trees and a large color dataset, enhancing explainability and accuracy over prior hand-crafted or learned features.

Findings

Achieved up to 80.4% precision on person search datasets.

Effectively models human color perception variability.

Provides explainable semantic color labels for pedestrians.

Abstract

We propose an explainable model to generate semantic color labels for person search. In this context, persons are described from their semantic parts, such as hat, shirt, etc. Person search consists in looking for people based on these descriptions. In this work, we aim to improve the accuracy of color labels for people. Our goal is to handle the high variability of human perception. Existing solutions are based on hand-crafted features or learnt features that are not explainable. Moreover most of them only focus on a limited set of colors. We propose a method based on binary search trees and a large peer-labelled color name dataset. This allows us to synthesize the human perception of colors. Using semantic segmentation and our color labeling method, we label segments of pedestrians with their associated colors. We evaluate our solution on person search on datasets such as PCN, and show a precision as high as 80.4%.